I'm trying to convert numbers to binary and then fwrite the binary numbers to binary file.
Assuming all numbers are 7 bits numbers (int numbers < 127).
So in the end, the file will contain the numbers as blocks of 7 bits
I know that each BYTE is 8 bits, and i can't to write each number to 1 BYTE, but need to use the whole BYTE (i.e some of the numbers will be in 2 different BYTES)
if : 120 = 1111000 | 7 = 0000111 | 64 = 1000000
so the bit stream is 111100000001111000000 and should be written as
1111000|0 000111|10 00000|00
1BYTE 2BYTE 3BYTE
I thought to use a buffer
shifting 8 bits to buffer and then fwrite to the file, using pointers
But i just can't mange to write it. Thank you for your help
First, when dealing with bits use unsigned integers
unsigned char a = 0;
char bits[] = "1111000";
char *p = bits;
while (*p) {
a <<= 1; // shift left
a |= (*p == '1'); // add bit
p++; // next bit
}
So, you want to put 8 7-bit values in 7 8-bit locations
[-val-][-val-][-val-]...
00000001111111222222233333334444444555555566666667777777
[-loc8-][-loc8-][-loc8-]...
Just map bits and you're done
For example, the 3rd 7-bit value (assuming int val7[8]) can be written into the 8-bit locations (assuming int loc8[7]) with
loc8[1] &= 0xfc; // clear 2 bits
loc8[1] |= (val7[2] & 0x60) >> 5; // set 2 bits
loc8[2] &= 0x7; // clear 5 bits
loc8[2] |= (val7[2] & 0x1f) << 3; // set 5 bits
Related
I have an array with integer values from 0-511 (9 bits max). I am trying to write this to a file with fwrite.
For Example, with the array:
[257, 258, 259]
Which is 100000001, 100000010, 100000011
I am trying to write
100000001100000010100000011 + extra padding of 0s to the file
But since fwrite limits writes to 1 byte at a time, I am not sure how to go about doing this. I am new to bitwise operations and am not how to separate out the individual bytes.
You need a bit buffer.
Since you are writing 8 bits at the time, you must
have data type that can hold at least 9+7 bits at minimum. uint16_t would do,
but I recommend using size that would at least as big as your native int. Make sure you use unsigned types to avoid shifting issues.
uint32_t bitBuffer = 0; // Our temporary bit storage
uint32_t count = 0; // Number of bits in buffer
Let's assume we have single data:
uint32_t data9b = 257; // 1 0000 0001
Adding bits to buffer is simple; just shift bits at the end of the buffer,
and combine with OR.
bitBuffer |= (data9b << count); // At first iteration, shift does nothing
count += 9; // Update counter
After 9 bits are added, we can flush 8 bits to file.
while(count >= 8) {
writeToFile(bitBuffer & 0xFF); // Mask out lowest bits with AND
bitBuffer >>= 8; // Remove written bits
count -= 8; // Fix counter
}
After each cycle you have 0 - 7 bits left over in the buffer. At the end of all data, if you finish with non-multiple of 8 bits, just write remaining contents of bitBuffer to file.
if(count > 0)
writeToFile(bitBuffer);
Ok, so did it using bit shifting, oring (can also do with *, '+', % and /) but shift is more appropriate / readable, imo.
// Your data, n is the number of 9-bits values
uint16_t dat[] = { 257, 258, 259 };
int i,n = sizeof(dat)/sizeof(*dat);
// out file
FILE *fp = fopen("f8.bin","w");
uint16_t one = 0;
int shift = 0;
uint8_t b;
// main loop
for(i=0 ; i<n ; i++) {
b = (dat[i] << shift) | one; // one is remainder from previous value
one = dat[i]>>(8-shift); // Move the remaining MSb to the right
shift = (shift+9) % 8; // next shift
fwrite(&b, 1, 1, fp); // write our b byte
}
// remainder, always have a remainder
fwrite(&one, 1, 1, fp);
fclose(fp);
Had fun :-)
My data is 12-bit and is stored in an array of 16-bit values. They're just values < 4095.
I need to output the 12-bit data in 8-bit chunks; top row are the 12 bits of the input values, bottom row is the 8 bits of the output values.
11|10|09|08|07|06|05|04|03|02|01|00|11|10|09|08|07|06|05|04|03|02|01|00
07|06|05|04|03|02|01|00|07|06|05|04|03|02|01|00|07|06|05|04|03|02|01|00
So for the output array:
The first byte contains the first 8 bits of the first 12-bit value.
The second byte contains the last 4 bits of the first 12-bit value and the first 4 bits of the second 12-bit value.
The third byte contains the last 8 bits of the second value.
And so on...
So ideally I want to turn an array of 12-bit numbers stored in a 16-bit array into an 8-bit array where the values are contiguous.
Technically it doesn't have to be out as an 8-bit array, I can output the 8-bit values through a function SPI.Transfer(byte) as I step through the 16-bit array.
So this works; maybe there is a more elegant way?
byteCount = 0;
for (int i = 0; i < numberOf16bitItems; i++) {
if (i % 2) {
//if odd then just grab the last 8 bits
toArray[byteCount] = fromArray[i] & 0b11111111;
byteCount++;
} else {
// if even get the first 8 bits and output, plus get the last 4 bits and the next bits of the next item
toArray[byteCount] = (fromArray[i] >> 4) & 0b11111111;
byteCount++;
toArray[byteCount] = ((fromArray[i+1] >> 8) & 0b1111) | (fromArray[i] & 0b1111 << 4);
byteCount++;
}
}
You need to specify endianness and the terms first and last. First from MSBit or LSBit? One way or another there will be lots of shifting, oring, anding in bussiness. And your result data will be as in source format, endianness and first/last point of view.
Assuming first 8 bits are starting from most significant bit:
for(i = 0; i < Lenght_of_16_Bit_Array; i++)
{
if(i % 2 == 0)
i8 = a16[i] >> 4; // takes first 8-bit and chops least 4-bits
else
i8 = a16[i-1] << 4 | a16[i] >> 8 // takes least 4-bits of prior item and chops 8-bits of current item.
a8[i] = i8;
}
// last control if your initial length is an odd number(means the index is even)
if(Lenght_of_16_Bit_Array % 2 = 1)
a8[Lenght_of_16_Bit_Array] = a16[Lenght_of_16_Bit_Array - 1] & 0xF; //recover that last 4 bits
I'm making a program to receive a bunch of 0's and 1's with a µC and need to take any amount of bits (1 to 16) from any position.
I.E. I have 150 bits and I want to take 6 bits from the 32th bit and copy it to a char (8bits) variable; I know I can do it with strings by saving as ASCII 0's and 1's, but I have not a lot of RAM, so I need to save it as bits.
The bigger variable is a unsigned 32 bits long, but save the data is not my problem, the problem is how to access to a specific bits positions and copy that to a char(8) variable.
You can use bitwise operators:
//bits: your bits (byte array), start: index of the first bit of the char you want
char select(char* bits, int start) {
dec = start%8;
return bits[start/8]>>dec + bits[start/8+1]<<dec;
}
The code above supposed start < (bits.size()-8)
[EDIT]
You can change the char* to any type you want. However you will need to change dec value to the appropriate number of bits (8*SIZE_IN_BYTES) and then apply operator | ("logical or") to get your char back
example:
char select(int* bits, int start) {
nbitsint = 8*4;
dec = start%nbitsint;
if (dec < nbitsint-8) {
// | 0xff creates creates a byte
return (bits[start/nbitsint]>>((3-dec/8)*8+dec%8) | 0xff;
}
// Getting a byte which is astride two values is tricky
return (bits[start/nbitsint]>>(start%8) + bits[start/nbitsint+1]<<(start%8) | 0xff;
}
I'm writing an algorithm that compresses data (LZSS) and it requires me to have two 13-bit values which I'll have to later merge together.
In some cases, however, I don't need 13 bits; 8 are enough.
For this purpose I have a structure like this:
typedef struct pattern
{
char is_compressed:1; //flag
short index :13; //first value
short length :13; //second value
unsigned char c; //is 8 bits are enough, use this instead
} Pattern;
I therefore have an array of these structures, and each structure can either contain the two 13-bit values or an 8-bit value.
I am now looping over this array, and my objective is to merge all these bits together.
I easily calculated the total number of bits used and the number of arrays of unsigned chars (8 bits) needed in order to store all the values:
int compressed = 0, plain = 0;
//count is the amount of patterns i have and p is the array of patterns (the structures)
for (int i = 0; i < count; i++)
{
if (p[i]->is_compressed)
compressed++;
else
plain++;
}
//this stores the number of bits used in the pattern (13 for length and 13 for the index or 8 for the plain uchar)
int tot_bits = compressed * 26 + plain * 8;
//since we can only write a minimum of 8 bits, we calculate how many arrays are needed to store the bits
int nr_of_arrays = (tot_bits % 8 == 0) ? tot_bits / 8 : (tot_bits / 8) + 1;
//we allocate the needed memory for the array of unsigned chars that will contain, concatenated, all the bits
unsigned char* uc = (unsigned char*) malloc(nr_of_arrays * sizeof(unsigned char));
After allocating the memory for the array I'm going to fill, I simply loop through the array of structures and recognize whether the structure I'm looking at contains the two 13-bit values or just the 8-bit one
for (int i = 0; i < count; i++)
{
if (p->is_compressed)
{
//The structure contains the two 13 bits value
}
else
{
//The structure only contains the 8 bits value
}
}
Here I'm stuck and can't seem to figure out a proper way of getting the job done.
Does anybody of you know how to implement that part there?
A practical example would be:
pattern 1 contains the 2 13-bit values:
1111 1111 1111 1
0000 0000 0000 0
pattern 2 contains the 8-bit value
1010 1010
total bits: 34
number of arrays required: 5 (that will waste 6 bits)
resulting array is:
[0] 1111 1111
[1] 1111 1000
[2] 0000 0000
[3] 0010 1010
[4] 1000 0000 (the remaining 6 bits are set to 0)
One way to do that is to write bytes one by one and keep track of partial bytes as you write.
You need a pointer to your char array, and an integer to keep track of how many bits you wrote to the last byte. Every time you write bits, you check how many bits you can write to the last byte, and you write these bits accordingly (ex: if there is 5 bits free, you shift your next value by 3 and add it to the last byte). Every time a byte is complete, you increment your array pointer and reset your bit tracker.
A clean way to implement this would be to write functions like :
void BitWriter_init( char *myArray );
void BitWriter_write( int theBitsToWrite, int howManyBits );
Now you just have to figure out how to implement these functions, or use any other method of your choice.
The problem intrigued me. Here's a possible implementation of "by using a lot of bitwise operations":
/* A writable bit string, with an indicator of the next available bit */
struct bitbuffer {
uint8_t *bytes;
size_t next_bit;
};
/*
* writes the bits represented by the given pattern to the next available
* positions in the specified bit buffer
*/
void write_bits(struct bitbuffer *buffer, Pattern *pattern) {
/* The index of the byte containing the next available bit */
size_t next_byte = buffer->next_bit / 8;
/* the number of bits already used in the next available byte */
unsigned bits_used = buffer->next_bit % 8;
if (pattern->is_compressed) {
/* assemble the bits to write in a 32-bit block */
uint32_t bits = pattern->index << 13 + pattern->length;
if (bits_used == 7) {
/* special case: the bits to write will span 5 bytes */
/* the first bit written will be the last in the current byte */
uint8_t first_bit = bits >> 25;
buffer->bytes[next_byte] |= first_bit;
/* write the next 8 bits to the next byte */
buffer->bytes[++next_byte] = (bits >> 17) & 0xFF;
/* align the tail of the bit block with the buffer*/
bits <<= 7;
} else {
/* the first bits written will fill out the current byte */
uint8_t first_bits = (bits >> (18 + bits_used)) & 0xFF;
buffer->bytes[next_byte] |= first_bits;
/* align the tail of the bit block with the buffer*/
bits <<= (6 - bits_used);
}
/*
* Write the remainder of the bit block to the buffer,
* most-significant bits first. Three (more) bytes will be modified.
*/
buffer->bytes[++next_byte] = (bits >> 16) & 0xFF;
buffer->bytes[++next_byte] = (bits >> 8) & 0xFF;
buffer->bytes[++next_byte] = bits & 0xFF;
/* update the buffer's index of the next available bit */
buffer->next_bit += 26;
} else { /* the pattern is not compressed */
if (bits_used) {
/* the bits to write will span two bytes in the buffer */
buffer->bytes[next_byte] |= (pattern->c >> bits_used);
buffer[++next_byte] = (pattern->c << bits_used) & 0xFF;
} else {
/* the bits to write exactly fill the next buffer byte */
buffer->bytes[next_byte] = pattern->c;
}
/* update the buffer's index of the next available bit */
buffer->next_bit += 8;
}
}
I need to read 32bit instructions from a binary file.
so what i have right now is:
unsigned char buffer[4];
fread(buffer,sizeof(buffer),1,file);
which will put 4 bytes in an array
how should I approach that to connect those 4 bytes together in order to process 32bit instruction later?
Or should I even start in a different way and not use fread?
my weird method right now is to create an array of ints of size 32 and the fill it with bits from buffer array
The answer depends on how the 32-bit integer is stored in the binary file. (I'll assume that the integer is unsigned, because it really is an id, and use the type uint32_t from <stdint.h>.)
Native byte order The data was written out as integer on this machine. Just read the integer with fread:
uint32_t op;
fread(&op, sizeof(op), 1, file);
Rationale: fread read the raw representation of the integer into memory. The matching fwrite does the reverse: It writes the raw representation to thze file. If you don't need to exchange the file between platforms, this is a good method to store and read data.
Little-endian byte order The data is stored as four bytes, least significant byte first:
uint32_t op = 0u;
op |= getc(file); // 0x000000AA
op |= getc(file) << 8; // 0x0000BBaa
op |= getc(file) << 16; // 0x00CCbbaa
op |= getc(file) << 24; // 0xDDccbbaa
Rationale: getc reads a char and returns an integer between 0 and 255. (The case where the stream runs out and getc returns the negative value EOF is not considered here for brevity, viz laziness.) Build your integer by shifting each byte you read by multiples of 8 and or them with the existing value. The comments sketch how it works. The capital letters are being read, the lower-case letters were already there. Zeros have not yet been assigned.
Big-endian byte order The data is stored as four bytes, least significant byte last:
uint32_t op = 0u;
op |= getc(file) << 24; // 0xAA000000
op |= getc(file) << 16; // 0xaaBB0000
op |= getc(file) << 8; // 0xaabbCC00
op |= getc(file); // 0xaabbccDD
Rationale: Pretty much the same as above, only that you shift the bytes in another order.
You can imagine little-endian and big-endian as writing the number one hundred and twenty tree (CXXIII) as either 321 or 123. The bit-shifting is similar to shifting decimal digtis when dividing by or multiplying with powers of 10, only that you shift my 8 bits to multiply with 2^8 = 256 here.
Add
unsigned int instruction;
memcpy(&instruction,buffer,4);
to your code. This will copy the 4 bytes of buffer to a single 32-bit variable. Hence you will get connected 4 bytes :)
If you know that the int in the file is the same endian as the machine the program's running on, then you can read straight into the int. No need for a char buffer.
unsigned int instruction;
fread(&instruction,sizeof(instruction),1,file);
If you know the endianness of the int in the file, but not the machine the program's running on, then you'll need to add and shift the bytes together.
unsigned char buffer[4];
unsigned int instruction;
fread(buffer,sizeof(buffer),1,file);
//big-endian
instruction = (buffer[0]<<24) + (buffer[1]<<16) + (buffer[2]<<8) + buffer[3];
//little-endian
instruction = (buffer[3]<<24) + (buffer[2]<<16) + (buffer[1]<<8) + buffer[0];
Another way to think of this is that it's a positional number system in base-256. So just like you combine digits in a base-10.
257
= 2*100 + 5*10 + 7
= 2*10^2 + 5*10^1 + 7*10^0
So you can also combine them using Horner's rule.
//big-endian
instruction = ((((buffer[0]*256) + buffer[1]*256) + buffer[2]*256) + buffer[3]);
//little-endian
instruction = ((((buffer[3]*256) + buffer[2]*256) + buffer[1]*256) + buffer[0]);
#luser droog
There are two bugs in your code.
The size of the variable "instruction" must not be 4 bytes: for example, Turbo C assumes sizeof(int) to be 2. Obviously, your program fails in this case. But, what is much more important and not so obvious: your program will also fail in case sizeof(int) be more than 4 bytes! To understand this, consider the following example:
int main()
{ const unsigned char a[4] = {0x21,0x43,0x65,0x87};
const unsigned char* p = &a;
unsigned long x = (((((p[3] << 8) + p[2]) << 8) + p[1]) << 8) + p[0];
printf("%08lX\n", x);
return 0;
}
This program prints "FFFFFFFF87654321" under amd64, because an unsigned char variable becomes SIGNED INT when it is used! So, changing the type of the variable "instruction" from "int" to "long" does not solve the problem.
The only way is to write something like:
unsigned long instruction;
instruction = 0;
for (int i = 0, unsigned char* p = buffer + 3; i < 4; i++, p--) {
instruction <<= 8;
instruction += *p;
}