Combining 17bit data into byte array - c

I'm having a bit of an issue with trying to move groups of 17bit data in to a byte array. I don't want to have to go through step-by-step, but I can't figure out a logical loop.
I need it this way because I'm meant to calculate a checksum by adding up the all the byte values after combining them like this.
So here is what I am struggling with.
I have 16 byte arrays. The first 3 bytes of the array contain the 17 bits I'm after. (8 bits from [0], 8 bits from [1], and the MSB from [2].)
I need to move these 16 17bit values to one separate byte array.
The first one is easy:
int index = 0;
myArray[index++] = driverData[driver][0]; //First byte
myArray[index++] = driverData[driver][1]; //Second byte
myArray[index] = (driverData[driver][2] & 0x80) << 7; //First bit of the third byte.
From here though it gets harder to attempt any kind of loop to move these over.
driver++;<br>
//Take the 7 MSBs from the data array.
myArray[index++] |= (byte)(driverData[driver][0] & 0x7e >> 1);
//This leaves a single bit left over on driverData[driver][0].
myArray[index] = (byte)(driverData[driver][1] & 0x1 << 7);
I think you get the picture. Am I doing this all wrong? Can anyone point me in the right direction?

Sounds like you have a prime number loop large enough to make coding the individual cases a bad idea. This is a classic packing problem. You need a loop that iterates through your destination, and some inner code that gets more bits to pack. Your packing code should know how many bits are available to it from the last iteration, how many it needs, and should be able to increment the source pointer if it doesn't have enough.

OK, so this looks to be working. I probably need to test it more, but this seems to be giving me the result I expect so far. I'm sure I could do this better somehow.
// ... //
void foo()
{
//Lets start by getting all the 17bit values from each driver for the board.
int bitIndex = 7;
int byteIndex = 0;
int stopIndex = chipIndex + GetChipCount();
//Now we start the shiftyness.
for (int driver = chipIndex; driver < stopIndex; driver++) {
int userBits =
(driverData[driver][0] & 0xff) << 9 | (driverData[driver][1]
& 0xff)
<< 1 | (driverData[driver][2] & 0x80) >> 7;
AddBitsToArray(userBits, ref bitIndex, ref byteIndex);
}
}
/// <summary>
/// Takes the 17 bits, and adds them to the byte array.
/// </summary>
private void AddBitsToArray(int userBits, ref int bitIndex, ref int byteIndex)
{
int bitCount = 17;
while (bitCount > 0) {
//First 8 bytes.
checksumBytes[byteIndex] |=
(byte) (((userBits & bitValue(bitCount - 1)) >>
(bitCount - 1)) << bitIndex);
//Move up the bit index to be written to.
bitIndex--;
//Decrement the number of bits left to shift.
bitCount--;
//If we have gone past the 8th bit, reset the bitIndex and increment the byteIndex.
if (bitIndex >= 0)
continue;
bitIndex = 7;
byteIndex++;
}
}
/// <summary>
/// Returns the value of a single bit at the given index.
/// </summary>
private int bitValue(int bitIndex)
{
return (int)(Math.Pow(2, bitIndex));
}

Here is what I came up with. The first part of the method is just setting up some fake input data, so remove that and add arguments as needed. The OutputData array is unnecessarily large but I didn't spend time to calculate its actual length.
I used 170 as the input value which is 10101010 and was helpful in validation.
private void BitShift17()
{
const int NumChunks = 16;
byte[] DriverData = new byte[]
{
170,
170,
170
};
byte[][] InputData = new byte[NumChunks][];
for (int n = 0; n < NumChunks; n++)
InputData[n] = DriverData;
byte[] OutputData = new byte[NumChunks * 3]; // Unnecessarily large
int OutputIndex = 0;
int BitPosition = 0;
for (int Driver = 0; Driver < InputData.Length; Driver++)
{
for (int InputIndex = 0; InputIndex < 3; InputIndex++)
{
byte InputByte = InputIndex == 2 ? (byte)(InputData[Driver][InputIndex] & 128) : InputData[Driver][InputIndex];
if (BitPosition == 0)
{
OutputData[OutputIndex] = InputByte;
if (InputIndex == 2)
BitPosition++;
else
OutputIndex++;
}
else
{
if (InputIndex == 2)
{
OutputData[OutputIndex] |= (byte)(InputByte >> BitPosition);
BitPosition++;
}
else
{
OutputData[OutputIndex] |= (byte)(InputByte >> BitPosition);
OutputIndex++;
OutputData[OutputIndex] = (byte)(InputByte << 8 - BitPosition);
}
}
}
if (BitPosition > 7) BitPosition = 0;
}
}

Related

Efficient algorithm for finding a byte in a bit array

Given a bytearray uint8_t data[N] what is an efficient method to find a byte uint8_t search within it even if search is not octet aligned? i.e. the first three bits of search could be in data[i] and the next 5 bits in data[i+1].
My current method involves creating a bool get_bit(const uint8_t* src, struct internal_state* state) function (struct internal_state contains a mask that is bitshifted right, &ed with src and returned, maintaining size_t src_index < size_t src_len) , leftshifting the returned bits into a uint8_t my_register and comparing it with search every time, and using state->src_index and state->src_mask to get the position of the matched byte.
Is there a better method for this?
If you're searching an eight bit pattern within a large array you can implement a sliding window over 16 bit values to check if the searched pattern is part of the two bytes forming that 16 bit value.
To be portable you have to take care of endianness issues which is done by my implementation by building the 16 bit value to search for the pattern manually. The high byte is always the currently iterated byte and the low byte is the following byte. If you do a simple conversion like value = *((unsigned short *)pData) you will run into trouble on x86 processors...
Once value, cmp and mask are setup cmp and mask are shifted. If the pattern was not found within hi high byte the loop continues by checking the next byte as start byte.
Here is my implementation including some debug printouts (the function returns the bit position or -1 if pattern was not found):
int findPattern(unsigned char *data, int size, unsigned char pattern)
{
int result = -1;
unsigned char *pData;
unsigned char *pEnd;
unsigned short value;
unsigned short mask;
unsigned short cmp;
int tmpResult;
if ((data != NULL) && (size > 0))
{
pData = data;
pEnd = data + size;
while ((pData < pEnd) && (result == -1))
{
printf("\n\npData = {%02x, %02x, ...};\n", pData[0], pData[1]);
if ((pData + 1) < pEnd) /* still at least two bytes to check? */
{
tmpResult = (int)(pData - data) * 8; /* calculate bit offset according to current byte */
/* avoid endianness troubles by "manually" building value! */
value = *pData << 8;
pData++;
value += *pData;
/* create a sliding window to check if search patter is within value */
cmp = pattern << 8;
mask = 0xFF00;
while (mask > 0x00FF) /* the low byte is checked within next iteration! */
{
printf("cmp = %04x, mask = %04x, tmpResult = %d\n", cmp, mask, tmpResult);
if ((value & mask) == cmp)
{
result = tmpResult;
break;
}
tmpResult++; /* count bits! */
mask >>= 1;
cmp >>= 1;
}
}
else
{
/* only one chance left if there is only one byte left to check! */
if (*pData == pattern)
{
result = (int)(pData - data) * 8;
}
pData++;
}
}
}
return (result);
}
I don't think you can do much better than this in C:
/*
* Searches for the 8-bit pattern represented by 'needle' in the bit array
* represented by 'haystack'.
*
* Returns the index *in bits* of the first appearance of 'needle', or
* -1 if 'needle' is not found.
*/
int search(uint8_t needle, int num_bytes, uint8_t haystack[num_bytes]) {
if (num_bytes > 0) {
uint16_t window = haystack[0];
if (window == needle) return 0;
for (int i = 1; i < num_bytes; i += 1) {
window = window << 8 + haystack[i];
/* Candidate for unrolling: */
for (int j = 7; j >= 0; j -= 1) {
if ((window >> j) & 0xff == needle) {
return 8 * i - j;
}
}
}
}
return -1;
}
The main idea is to handle the 87.5% of cases that cross the boundary between consecutive bytes by pairing bytes in a wider data type (uint16_t in this case). You could adjust it to use an even wider data type, but I'm not sure that would gain anything.
What you cannot safely or easily do is anything involving casting part or all of your array to a wider integer type via a pointer (i.e. (uint16_t *)&haystack[i]). You cannot be ensured of proper alignment for such a cast, nor of the byte order with which the result might be interpreted.
I don't know if it would be better, but i would use sliding window.
uint counter = 0, feeder = 8;
uint window = data[0];
while (search ^ (window & 0xff)){
window >>= 1;
feeder--;
if (feeder < 8){
counter++;
if (counter >= data.length) {
feeder = 0;
break;
}
window |= data[counter] << feeder;
feeder += 8;
}
}
//Returns index of first bit of first sequence occurrence or -1 if sequence is not found
return (feeder > 0) ? (counter+1)*8-feeder : -1;
Also with some alterations you can use this method to search for arbitrary length (1 to 64-array_element_size_in_bits) bits sequence.
If AVX2 is acceptable (with earlier versions it didn't work out so well, but you can still do something there), you can search in a lot of places at the same time. I couldn't test this on my machine (only compile) so the following is more to give to you an idea of how it could be approached than copy&paste code, so I'll try to explain it rather than just code-dump.
The main idea is to read an uint64_t, shift it right by all values that make sense (0 through 7), then for each of those 8 new uint64_t's, test whether the byte is in there. Small complication: for the uint64_t's shifted by more than 0, the highest position should not be counted since it has zeroes shifted into it that might not be in the actual data. Once this is done, the next uint64_t should be read at an offset of 7 from the current one, otherwise there is a boundary that is not checked across. That's fine though, unaligned loads aren't so bad anymore, especially if they're not wide.
So now for some (untested, and incomplete, see below) code,
__m256i needle = _mm256_set1_epi8(find);
size_t i;
for (i = 0; i < n - 6; i += 7) {
// unaligned load here, but that's OK
uint64_t d = *(uint64_t*)(data + i);
__m256i x = _mm256_set1_epi64x(d);
__m256i low = _mm256_srlv_epi64(x, _mm256_set_epi64x(3, 2, 1, 0));
__m256i high = _mm256_srlv_epi64(x, _mm256_set_epi64x(7, 6, 5, 4));
low = _mm256_cmpeq_epi8(low, needle);
high = _mm256_cmpeq_epi8(high, needle);
// in the qword right-shifted by 0, all positions are valid
// otherwise, the top position corresponds to an incomplete byte
uint32_t lowmask = 0x7f7f7fffu & _mm256_movemask_epi8(low);
uint32_t highmask = 0x7f7f7f7fu & _mm256_movemask_epi8(high);
uint64_t mask = lowmask | ((uint64_t)highmask << 32);
if (mask) {
int bitindex = __builtin_ffsl(mask);
// the bit-index and byte-index are swapped
return 8 * (i + (bitindex & 7)) + (bitindex >> 3);
}
}
The funny "bit-index and byte-index are swapped" thing is because searching within a qword is done byte by byte and the results of those comparisons end up in 8 adjacent bits, while the search for "shifted by 1" ends up in the next 8 bits and so on. So in the resulting masks, the index of the byte that contains the 1 is a bit-offset, but the bit-index within that byte is actually the byte-offset, for example 0x8000 would correspond to finding the byte at the 7th byte of the qword that was right-shifted by 1, so the actual index is 8*7+1.
There is also the issue of the "tail", the part of the data left over when all blocks of 7 bytes have been processed. It can be done much the same way, but now more positions contain bogus bytes. Now n - i bytes are left over, so the mask has to have n - i bits set in the lowest byte, and one fewer for all other bytes (for the same reason as earlier, the other positions have zeroes shifted in). Also, if there is exactly 1 byte "left", it isn't really left because it would have been tested already, but that doesn't really matter. I'll assume the data is sufficiently padded that accessing out of bounds doesn't matter. Here it is, untested:
if (i < n - 1) {
// make n-i-1 bits, then copy them to every byte
uint32_t validh = ((1u << (n - i - 1)) - 1) * 0x01010101;
// the lowest position has an extra valid bit, set lowest zero
uint32_t validl = (validh + 1) | validh;
uint64_t d = *(uint64_t*)(data + i);
__m256i x = _mm256_set1_epi64x(d);
__m256i low = _mm256_srlv_epi64(x, _mm256_set_epi64x(3, 2, 1, 0));
__m256i high = _mm256_srlv_epi64(x, _mm256_set_epi64x(7, 6, 5, 4));
low = _mm256_cmpeq_epi8(low, needle);
high = _mm256_cmpeq_epi8(high, needle);
uint32_t lowmask = validl & _mm256_movemask_epi8(low);
uint32_t highmask = validh & _mm256_movemask_epi8(high);
uint64_t mask = lowmask | ((uint64_t)highmask << 32);
if (mask) {
int bitindex = __builtin_ffsl(mask);
return 8 * (i + (bitindex & 7)) + (bitindex >> 3);
}
}
If you are searching a large amount of memory and can afford an expensive setup, another approach is to use a 64K lookup table. For each possible 16-bit value, the table stores a byte containing the bit shift offset at which the matching octet occurs (+1, so 0 can indicate no match). You can initialize it like this:
uint8_t* g_pLookupTable = malloc(65536);
void initLUT(uint8_t octet)
{
memset(g_pLookupTable, 0, 65536); // zero out
for(int i = 0; i < 65536; i++)
{
for(int j = 7; j >= 0; j--)
{
if(((i >> j) & 255) == octet)
{
g_pLookupTable[i] = j + 1;
break;
}
}
}
}
Note that the case where the value is shifted 8 bits is not included (the reason will be obvious in a minute).
Then you can scan through your array of bytes like this:
int findByteMatch(uint8_t* pArray, uint8_t octet, int length)
{
if(length >= 0)
{
uint16_t index = (uint16_t)pArray[0];
if(index == octet)
return 0;
for(int bit, i = 1; i < length; i++)
{
index = (index << 8) | pArray[i];
if(bit = g_pLookupTable[index])
return (i * 8) - (bit - 1);
}
}
return -1;
}
Further optimization:
Read 32 or however many bits at a time from pArray into a uint32_t and then shift and AND each to get byte one at a time, OR with index and test, before reading another 4.
Pack the LUT into 32K by storing a nybble for each index. This might help it squeeze into the cache on some systems.
It will depend on your memory architecture whether this is faster than an unrolled loop that doesn't use a lookup table.

Breaking apart bit patterns, shifting and creating new patterns

As part of a larger problem, I have to take some binary value: 00000000 11011110 (8)
Then, I have to:
Derive the bit count in this function - so I've done that by finding the place of the most sig fig.
Then store the first 6 numbers of this value into the value 128, such that it equals: 10011110
Then store the last 5 numbers of this value into the value 192, such that it equals: 11000011 10011110
The two bytes should be stored in some array, buffer[]
I have written this function however, position does not appear to initialise properly in gdb and the values are not outputting correctly. This is my attempt:
void create_value(unsigned short init_val, unsigned char buffer[])
{
// get the count
int position = 0;
while (init_val >>= 1)
position++;
// get total
int count = position++;
int start = 128;
for (int i = 0; i < 7; i++)
if (((1 << i) & init_val) != 0) start = start | 1 << i;
buffer[0] = start;
start = 192;
for (int i = 7; i < 11; i++) {
if (((1 << i) & init_val) !=0) start = start | 1 << i;
}
buf[1] = start;
}
After
while (init_val >>= 1)
position++;
init_val will be 0. When you later use
if (((1 << i) & init_val) != 0) start = start | 1 << i;
you will never change start.
So, after reading through what you're trying to do (which is pretty confusingly described), why don't you:
void create_value(unsigned short init_value, unsigned char buffer[])
{
buffer[0] = (init_value & 63) | 128;
buffer[1] = ((init_value >> 6) & 31) | 192;
return;
}
What this does: init_value & 63 masks off all but the lowest 6 bits in init_value, as you wanted. The | 128 then sets the most significant bit of the byte (IFF CHAR_BIT == 8, mind you).
(init_value >> 6) shifts init_value down by 6 bits, so now the original bits 6-11 are bits 0-4. & 31 masks off all bit the lowest 5 bits in this value, | 192 sets the two most significant bits.

Pushing Nibbles onto an integer stack in C

I have an unsigned integer, and I want to push nibbles into it. For example, if I have nibbles with values 1, 2, 3, 4, 5, 6, 7 & 8, I want to be able to push the first nibble into my integer to make:
0x10000000 (268435456)
After the second push, I will have:
0x12000000 (301989888)
After the third push, I will have:
0x12300000 (305135616)
And so forth. Does anyone have a neat and cunning idea for how I might achieve this? The solution will need to be able to be given any number as a starting point and push onto the first available zero. So providing int 301989888 as a starting point and pushing 3 will result in 305135616. Pushing from MSB or LSB would be useful as well.
My apologies. It sounds like an exam question. It isn't - I just want to try an experiment, and I'm stuck before I start!
The answer ticked is perfect! I've modified it slightly as follows (just to make it self contained), and I'm as a happy as a tick!
#define left 0
#define right 1
void push(unsigned* number, int nibble,int direction){
int i, shift;
if (direction){
for (i = 28; i >= 0; i -= 4){
if (!(*number & (0xfU << i)))
shift = i;
}
}
else{
for (i = 0; i <= 28; i += 4){
if (!(*number & (0xfU << i)))
shift = i;
}
}
*number|=nibble<<shift;
}
Called as follows:
push(&x,nibble,left);
My apologies for the formatting.
There are two things you need to do - detect where to put the next nibble, and then put it there. For detection, you can mask & shift:
int nextLocation(uint32_t x)
{
int i;
for (i = 28; i >= 0; i -= 4)
{
if (!(x & (0xfU << i)))
return i;
}
return -1;
}
This function will return the amount of upshift you need to "push" your next nibble (or -1 if your integer is already full).
Then, you need to put in the new value (assuming that x is the value you want to push into and nibble is the value you want to push):
int shiftAmount = nextLocation(x);
x |= nibble << shiftAmount;
To push the other direction, you can just change the direction of the for loop in the nextLocation function:
for (i = 0; i <= 28; i += 4)
Here is an overly simple example which does what you want. It "pushes," but not in an automated way (if that's what you're looking for). But this demonstrates the concept (note: I combined each nibble into a byte).
#include <iostream>
using namespace std;
int main()
{
int x = (0x12 << 24) | (0x34 << 16) | (0x56 << 8) | (0x78);
cout<< hex << x << endl;
return 0;
}

Bitwise shifting array of char's

I have got an array of chars that I'm trying to bitwise shift right >>, then & with another array. I think I have got the wrong idea of how to do this.
I thought, even though it was an array of chars just stating my_array >>= 1 would shift everything but I am getting an error: "error: invalid operands to binary >> (have ‘char[8]’ and ‘int’)"
The bitwise comparision I am trying to do is with a similar size array initiated to all "0's"...for that I'm getting: "error: invalid operands to binary & (have ‘char *’ and ‘char *’)"
Do I need to convert these array's into something else before I can shift and compare?
Sorry, I was not super clear... All great advice up to this point and I think I am realizing more that there is no super easy way to do this. More specifically, what I am trying to do is shift the bits of the WHOLE char array right 1, adding the bit shifted off the right back to the left most side of the array, do the bitwise compare with another array of same size.
Technically the compare doesn't have to be array with array... I just need the bits. Would it be easier to convert the array's to something else before trying to do the shifts/comparisons?
You have to shift and compare elementwise.
for(i = 0; i < len; ++i)
array[i] >>= 3;
for example. If you want to move the bits shifted out of one element to the next, it's more complicated, say you're shifting right, then
unsigned char bits1 = 0, bits2 = 0;
for(i = len-1; i >= 0; --i) {
bits2 = array[i] & 0x07;
array[i] >>= 3;
array[i] |= bits1 << 5;
bits1 = bits2;
}
traversing the array in the other direction because you need the bits from the next higher slot.
You'll have to shift the entries in the array one by one. (And if you want to compare two of these, you'll need to do it element by element.)
If you were hoping that bits shifted off each char would get shifted into the next one, you'll need to take care of that manually too.
If you are wanting that shift-into-the-next-byte behaviour, and don't mind making your code nasty and nonportable and bug-prone, you might be able to take a pointer to the array, cast it to something like unsigned long long *, dereference it and shift the resulting integer, and store it back again.
BUT if that's the behaviour you want then you should be using an integer instead of a char[8] to begin with.
(If you could say more about what you're actually aiming to achieve, then more helpful answers may be possible.)
If you want to perform operations such as shifting / OR / XOR / AND / etc.. on arrays, you should perform it in a loop, you cannot perform it directly on the array.
/** Shift an array right.
* #param ar The array to shift.
* #param size The number of array elements.
* #param shift The number of bits to shift.
*/
void shift_right(unsigned char *ar, int size, int shift)
{
int carry = 0; // Clear the initial carry bit.
while (shift--) { // For each bit to shift ...
for (int i = size - 1; i >= 0; --i) { // For each element of the array from high to low ...
int next = (ar[i] & 1) ? 0x80 : 0; // ... if the low bit is set, set the carry bit.
ar[i] = carry | (ar[i] >> 1); // Shift the element one bit left and addthe old carry.
carry = next; // Remember the old carry for next time.
}
}
}
You can shift only members of that arrays, a char (or an int). You can't shift an entire array. Shifting my_array tries to perform a shift operation on an array type (or a pointer to char) which is impossible. Do this instead:
for (i = 0; i < size; i++) {
my_array[i] >>= 1;
}
Also you must be careful with chars because they are usually signed, and a char containing a negative value will bring '1' from the left instead of zeros. So you better use unsigned chars.
EDIT:
The code above is simplistic. If you intended to shift right the array as a whole, not just each byte on its own, then you need to "manually" copy each LSB to the MSB of the byte to its right. Take a loop at the answer of Richard Pennington.
/**
* shift a number of bits to the right
*
* #param SRC the array to shift
* #param len the length of the array
* #param shift the number of consecutive bits to shift
*
*/
static void shift_bits_right(uint8_t SRC[], uint16_t len, uint32_t shift) {
uint32_t i = 0;
uint8_t start = shift / 8;
uint8_t rest = shift % 8;
uint8_t previous = 0;
for(i = 0; i < len; i++) {
if(start <= i) {
previous = SRC[i - start];
}
uint8_t value = (previous << (8 - rest)) | SRC[i + start] >> rest;
SRC[i + start] = value;
}
}
I know this is old topic but i was not satisfied with the answers available, here is something i wrote recently which allows you to specify the amount of bits you can shift by and also there is simple XOR encryption in it.
//https://github.com/ashvin-bhuttoo/CryptoTest/blob/master/CryptoTest/Crypto.cpp
//CRYPTO CONFIGURATION PARAMETERS
#define BIT_SHIFT 3
#define XOR_KEY 0x3C
#define ENABLE_XOR_VARIANCE true
////////////////////////////////
int get_rs_mask(int shift)
{
switch (shift)
{
case 0:
return 0x00;
case 1:
return 0x01;
case 2:
return 0x03;
case 3:
return 0x07;
case 4:
return 0x0F;
case 5:
return 0x1F;
case 6:
return 0x3F;
case 7:
return 0x7F;
default:
throw "get_rs_mask -> Error, shift argument outside legal range 0-7";
}
}
void shift_right(char* buf, int msg_len, int shift)
{
unsigned char tmp = 0x00, tmp2 = 0x00;
for (int k = 0; k <= msg_len; k++)
{
if (k == 0)
{
tmp = buf[k];
buf[k] >>= shift;
}
else
{
tmp2 = buf[k];
buf[k] >>= shift;
buf[k] |= ((tmp & get_rs_mask(shift)) << (8 - shift));
if (k != msg_len)
tmp = tmp2;
}
}
}
int get_ls_mask(int shift)
{
switch (shift)
{
case 0:
return 0x00;
case 1:
return 0x80;
case 2:
return 0xC0;
case 3:
return 0xE0;
case 4:
return 0xF0;
case 5:
return 0xF8;
case 6:
return 0xFC;
case 7:
return 0xFE;
default:
throw "get_ls_mask -> Error, shift argument outside legal range 0-7";
}
}
void shift_left(char* buf, int msg_len, int shift)
{
char tmp = 0x00, tmp2 = 0x00;
for (int k = msg_len; k >= 0; k--)
{
if (k == msg_len)
{
tmp = buf[k];
buf[k] <<= shift;
}
else
{
tmp2 = buf[k];
buf[k] <<= shift;
buf[k] |= ((tmp & get_ls_mask(shift)) >> (8 - shift));
tmp = tmp2;
}
}
}
void crypt(char* buf, int msg_len, bool decrypt = false)
{
if (!decrypt)
{
shift_right(buf, msg_len, BIT_SHIFT);
for (int k = 0; k < msg_len; k++)
{
buf[k] = buf[k] ^ XOR_KEY ^ k * (ENABLE_XOR_VARIANCE ? 2 : 0);
}
buf[msg_len] = '\0';
}
else
{
for (int k = 0; k < msg_len; k++)
{
buf[k] = buf[k] ^ XOR_KEY ^ k * (ENABLE_XOR_VARIANCE ? 2 : 0);
}
shift_left(buf, (msg_len)-1, BIT_SHIFT);
}
}
/**
* Shift a number of bits to the right
*
* #param array The array to shift
* #param len The length of the array
* #param shift The number of consecutive bits to shift. To the right if shift is positif.
*
*/
static void shift_bits_right(uint8_t *array, int len, int shift) {
uint8_t macro_shift = shift / 8;
shift = shift % 8;
uint8_t array_out[len];
memset(array_out, 0, len);
for(int i = 0; i < len; i++) {
if(i+macro_shift < len)
array_out[i+macro_shift] += array[i]>>shift;
if(i+macro_shift+1 < len)
array_out[i+macro_shift+1] += array[i]<<(8-shift);
}
memcpy(array, array_out, len);
}
For everyone who is looking for a code snippet to (logically) shift right a byte array that actually works:
template<size_t N> void shift_right(array<uint8_t, N>& arr, uint64_t bits)
{
int64_t num_bytes = bits / 8;
int64_t num_bits = bits % 8;
for(int64_t i = N-1; i >= 0; i--)
{
int64_t i_from = i - num_bytes;
int64_t i_from_minus_one = i - num_bytes - 1;
uint8_t v_from = i_from < 0 ? 0 : arr[i_from];
uint8_t v_from_minus_one = i_from_minus_one < 0 ? 0 : arr[i_from_minus_one];
arr[i] = v_from >> num_bits | v_from_minus_one << (8 - num_bits);
}
}

Efficient bitshifting an array of int?

To be on the same page, let's assume sizeof(int)=4 and sizeof(long)=8.
Given an array of integers, what would be an efficient method to logically bitshift the array to either the left or right?
I am contemplating an auxiliary variable such as a long, that will compute the bitshift for the first pair of elements (index 0 and 1) and set the first element (0). Continuing in this fashion the bitshift for elements (index 1 and 2) will be computer, and then index 1 will be set.
I think this is actually a fairly efficient method, but there are drawbacks. I cannot bitshift greater than 32 bits. I think using multiple auxiliary variables would work, but I'm envisioning recursion somewhere along the line.
There's no need to use a long as an intermediary. If you're shifting left, start with the highest order int, shifting right start at the lowest. Add in the carry from the adjacent element before you modify it.
void ShiftLeftByOne(int * arr, int len)
{
int i;
for (i = 0; i < len - 1; ++i)
{
arr[i] = (arr[i] << 1) | ((arr[i+1] >> 31) & 1);
}
arr[len-1] = arr[len-1] << 1;
}
This technique can be extended to do a shift of more than 1 bit. If you're doing more than 32 bits, you take the bit count mod 32 and shift by that, while moving the result further along in the array. For example, to shift left by 33 bits, the code will look nearly the same:
void ShiftLeftBy33(int * arr, int len)
{
int i;
for (i = 0; i < len - 2; ++i)
{
arr[i] = (arr[i+1] << 1) | ((arr[i+2] >> 31) & 1);
}
arr[len-2] = arr[len-1] << 1;
arr[len-1] = 0;
}
For anyone else, this is a more generic version of Mark Ransom's answer above for any number of bits and any type of array:
/* This function shifts an array of byte of size len by shft number of
bits to the left. Assumes array is big endian. */
#define ARR_TYPE uint8_t
void ShiftLeft(ARR_TYPE * arr_out, ARR_TYPE * arr_in, int arr_len, int shft)
{
const int int_n_bits = sizeof(ARR_TYPE) * 8;
int msb_shifts = shft % int_n_bits;
int lsb_shifts = int_n_bits - msb_shifts;
int byte_shft = shft / int_n_bits;
int last_byt = arr_len - byte_shft - 1;
for (int i = 0; i < arr_len; i++){
if (i <= last_byt){
int msb_idx = i + byte_shft;
arr_out[i] = arr_in[msb_idx] << msb_shifts;
if (i != last_byt)
arr_out[i] |= arr_in[msb_idx + 1] >> lsb_shifts;
}
else arr_out[i] = 0;
}
}
Take a look at BigInteger implementation in Java, which internally stores data as an array of bytes. Specifically you can check out the funcion leftShift(). Syntax is the same as in C, so it wouldn't be too difficult to write a pair of funciontions like those. Take into account too, that when it comes to bit shifting you can take advange of unsinged types in C. This means that in Java to safely shift data without messing around with sign you usually need bigger types to hold data (i.e. an int to shift a short, a long to shift an int, ...)

Resources