This may be a slightly theoretical question. I have a char array of bytes containing network packets. I want to check for the occurrence of a particular pair of bits ('01' or '10')every 66 bits. That is to say once I locate the first pair of bits I have to skip 66 bits and check the presence of same pair of bits again. I am trying to implement a program with masks and shifts and it is kind of getting complicated. I want to know if someone can suggest a better way to do the same thing.
The code I have written so far looks something like this. It is not complete though.
test_sync_bits(char *rec, int len)
{
uint8_t target_byte = 0;
int offset = 0;
int save_offset = 0;
uint8_t *pload = (uint8_t*)(rec + 24);
uint8_t seed_mask = 0xc0;
uint8_t seed_shift = 6;
uint8_t value = 0;
uint8_t found_sync = 0;
const uint8_t sync_bit_spacing = 66;
/*hunt for the first '10' or '01' combination.*/
target_byte = *(uint8_t*)(pload + offset);
/*Get all combinations of two bits from target byte.*/
while(seed_shift)
{
value = ((target_byte & seed_mask) >> seed_shift);
if((value == 0x01) || (value == 0x10))
{
save_offset = offset;
found_sync = 1;
break;
}
else
{
seed_mask = (seed_mask >> 2) ;
seed_shift-=2;
}
}
offset = offset + 8;
seed_shift = (seed_shift - 4) > 0 ? (seed_shift - 4) : (seed_shift + 8 - 4);
seed_mask = (seed_mask >> (6 - seed_shift));
}
Another idea I came up with was to use a structure defined below
typedef struct
{
int remainder_bits;
int extra_bits;
int extra_byte;
}remainder_bits_extra_bits_map_t;
static remainder_bits_extra_bits_map_t sync_bit_check [] =
{
{6, 4, 0},
{5, 5, 0},
{4, 6, 0},
{3, 7, 0},
{2, 8, 0},
{1, 1, 1},
{0, 2, 1},
};
Is my approach correct? Can anyone suggest any improvements for the same?
Lookup Table Idea
There are only 256 possible bytes. That is few enough that you can construct a lookup table of all the possible bit combinations that can happen in one byte.
The lookup table value could record the bit position of the pattern and it could also have special values that mark possible continuation start or continuation finish values.
Edit:
I decided that continuation values would be silly. Instead, to check for a pattern that overlaps a byte, shift the byte and OR in the bit from the other byte, or manually check the end bits at each byte. Maybe ((bytes[i] & 0x01) & (bytes[i+1] & 0x80)) == 0x80 and ((bytes[i] & 0x01) & (bytes[i+1] & 0x80)) == 0x01 would work for you.
You didn't say so I am also assuming that you are looking for the first match in any byte. If you are looking for every match, then checking for the end pattern at +66 bits, that's a different problem.
To create the lookup table, I would write a program to do it for me. It could be in your favorite script language or it could be in C. The program would write a file that looked something like:
/* each value is the bit position of a possible pattern OR'd with a pattern ID bit. */
/* 0 is no match */
#define P_01 0x00
#define P_10 0x10
const char byte_lookup[256] = {
/* 0: 0000_0000, 0000_0001, 0000_0010, 0000_0011 */
0, 2|P_01, 3|P_01, 3|P_01,
/* 4: 0000_0100, 0000_0101, 0000_0110, 0000_0111, */
4|P_01, 4|P_01, 4|P_01, 4|P_01,
/* 8: 0000_1000, 0000_1001, 0000_1010, 0000_1011, */
5|P_01, 5|P_01, 5|P_01, 5|P_01,
};
Tedious. That's why I would write a program to write it for me.
This is a variation of the classic de-blocking problem that often comes up when reading from a stream. That is, data comes in discrete units that don't match up to the unit size that you wish to scan. The challenges in this are 1) buffering (which doesn't affect you because you have access to the whole array) and 2) managing all of the state (as you found out). A good approach is to write a consumer function that acts something like fread() and fseek() which maintains its own state. It returns the requested data you're interested in, aligned properly to the buffers you give it.
Related
So I have to find the set bits (on 1) of an unsigned char variable in C?
A similar question is How to count the number of set bits in a 32-bit integer? But it uses an algorithm that's not easily adaptable to 8-bit unsigned chars (or its not apparent).
The algorithm suggested in the question How to count the number of set bits in a 32-bit integer? is trivially adapted to 8 bit:
int NumberOfSetBits( uint8_t b )
{
b = b - ((b >> 1) & 0x55);
b = (b & 0x33) + ((b >> 2) & 0x33);
return (((b + (b >> 4)) & 0x0F) * 0x01);
}
It is simply a case of shortening the constants the the least significant eight bits, and removing the final 24 bit right-shift. Equally it could be adapted for 16bit using an 8 bit shift. Note that in the case for 8 bit, the mechanical adaptation of the 32 bit algorithm results in a redundant * 0x01 which could be omitted.
The fastest approach for an 8-bit variable is using a lookup table.
Build an array of 256 values, one per 8-bit combination. Each value should contain the count of bits in its corresponding index:
int bit_count[] = {
// 00 01 02 03 04 05 06 07 08 09 0a, ... FE FF
0, 1, 1, 2, 1, 2, 2, 3, 1, 2, 2, ..., 7, 8
};
Getting a count of a combination is the same as looking up a value from the bit_count array. The advantage of this approach is that it is very fast.
You can generate the array using a simple program that counts bits one by one in a slow way:
for (int i = 0 ; i != 256 ; i++) {
int count = 0;
for (int p = 0 ; p != 8 ; p++) {
if (i & (1 << p)) {
count++;
}
}
printf("%d, ", count);
}
(demo that generates the table).
If you would like to trade some CPU cycles for memory, you can use a 16-byte lookup table for two 4-bit lookups:
static const char split_lookup[] = {
0, 1, 1, 2, 1, 2, 2, 3, 1, 2, 2, 3, 2, 3, 3, 4
};
int bit_count(unsigned char n) {
return split_lookup[n&0xF] + split_lookup[n>>4];
}
Demo.
I think you are looking for Hamming Weight algorithm for 8bits?
If it is true, here is the code:
unsigned char in = 22; //This is your input number
unsigned char out = 0;
in = in - ((in>>1) & 0x55);
in = (in & 0x33) + ((in>>2) & 0x33);
out = ((in + (in>>4) & 0x0F) * 0x01) ;
Counting the number of digits different than 0 is also known as a Hamming Weight. In this case, you are counting the number of 1's.
Dasblinkenlight provided you with a table driven implementation, and Olaf provided you with a software based solution. I think you have two other potential solutions. The first is to use a compiler extension, the second is to use an ASM specific instruction with inline assembly from C.
For the first alternative, see GCC's __builtin_popcount(). (Thanks to Artless Noise).
For the second alternative, you did not specify the embedded processor, but I'm going to offer this in case its ARM based.
Some ARM processors have the VCNT instruction, which performs the count for you. So you could do it from C with inline assembly:
inline
unsigned int hamming_weight(unsigned char value) {
__asm__ __volatile__ (
"VCNT.8"
: "=value"
: "value"
);
return value;
}
Also see Fastest way to count number of 1s in a register, ARM assembly.
For completeness, here is Kernighan's bit counting algorithm:
int count_bits(int n) {
int count = 0;
while(n != 0) {
n &= (n-1);
count++;
}
return count;
}
Also see Please explain the logic behind Kernighan's bit counting algorithm.
I made an optimized version. With a 32-bit processor, utilizing multiplication, bit shifting and masking can make smaller code for the same task, especially when the input domain is small (8-bit unsigned integer).
The following two code snippets are equivalent:
unsigned int bit_count_uint8(uint8_t x)
{
uint32_t n;
n = (uint32_t)(x * 0x08040201UL);
n = (uint32_t)(((n >> 3) & 0x11111111UL) * 0x11111111UL);
/* The "& 0x0F" will be optimized out but I add it for clarity. */
return (n >> 28) & 0x0F;
}
/*
unsigned int bit_count_uint8_traditional(uint8_t x)
{
x = x - ((x >> 1) & 0x55);
x = (x & 0x33) + ((x >> 2) & 0x33);
x = ((x + (x >> 4)) & 0x0F);
return x;
}
*/
This produces smallest binary code for IA-32, x86-64 and AArch32 (without NEON instruction set) as far as I can find.
For x86-64, this doesn't use the fewest number of instructions, but the bit shifts and downcasting avoid the use of 64-bit instructions and therefore save a few bytes in the compiled binary.
Interestingly, in IA-32 and x86-64, a variant of the above algorithm using a modulo ((((uint32_t)(x * 0x08040201U) >> 3) & 0x11111111U) % 0x0F) actually generates larger code, due to a requirement to move the remainder register for return value (mov eax,edx) after the div instruction. (I tested all of these in Compiler Explorer)
Explanation
I denote the eight bits of the byte x, from MSB to LSB, as a, b, c, d, e, f, g and h.
abcdefgh
* 00001000 00000100 00000010 00000001 (make 4 copies of x
--------------------------------------- with appropriate
abc defgh0ab cdefgh0a bcdefgh0 abcdefgh bit spacing)
>> 3
---------------------------------------
000defgh 0abcdefg h0abcdef gh0abcde
& 00010001 00010001 00010001 00010001
---------------------------------------
000d000h 000c000g 000b000f 000a000e
* 00010001 00010001 00010001 00010001
---------------------------------------
000d000h 000c000g 000b000f 000a000e
... 000h000c 000g000b 000f000a 000e
... 000c000g 000b000f 000a000e
... 000g000b 000f000a 000e
... 000b000f 000a000e
... 000f000a 000e
... 000a000e
... 000e
^^^^ (Bits 31-28 will contain the sum of the bits
a, b, c, d, e, f, g and h. Extract these
bits and we are done.)
Maybe not the fastest, but straightforward:
int count = 0;
for (int i = 0; i < 8; ++i) {
unsigned char c = 1 << i;
if (yourVar & c) {
//bit n°i is set
//first bit is bit n°0
count++;
}
}
For 8/16 bit MCUs, a loop will very likely be faster than the parallel-addition approach, as these MCUs cannot shift by more than one bit per instruction, so:
size_t popcount(uint8_t val)
{
size_t cnt = 0;
do {
cnt += val & 1U; // or: if ( val & 1 ) cnt++;
} while ( val >>= 1 ) ;
return cnt;
}
For the incrementation of cnt, you might profile. If still too slow, an assember implementation might be worth a try using carry flag (if available). While I am in against using assembler optimizations in general, such algorithms are one of the few good exceptions (still just after the C version fails).
If you can omit the Flash, a lookup table as proposed by #dasblinkenlight is likey the fastest approach.
Just a hint: For some architectures (notably ARM and x86/64), gcc has a builtin: __builtin_popcount(), you also might want to try if available (although it takes int at least). This might use a single CPU instruction - you cannot get faster and more compact.
Allow me to post a second answer. This one is the smallest possible for ARM processors with Advanced SIMD extension (NEON). It's even smaller than __builtin_popcount() (since __builtin_popcount() is optimized for unsigned int input, not uint8_t).
#ifdef __ARM_NEON
/* ARM C Language Extensions (ACLE) recommends us to check __ARM_NEON before
including <arm_neon.h> */
#include <arm_neon.h>
unsigned int bit_count_uint8(uint8_t x)
{
/* Set all lanes at once so that the compiler won't emit instruction to
zero-initialize other lanes. */
uint8x8_t v = vdup_n_u8(x);
/* Count the number of set bits for each lane (8-bit) in the vector. */
v = vcnt_u8(v);
/* Get lane 0 and discard other lanes. */
return vget_lane_u8(v, 0);
}
#endif
Okay.. according to the title i am trying to figure out a way - function that returns the character that dominates in a string. I might be able to figure it out.. but it seems something is wrong with my logic and i failed on this. IF someome can come up with this without problems i will be extremelly glad thank you.
I say "in a string" to make it more simplified. I am actually doing that from a buffered data containing a BMP image. Trying to output the base color (the dominant pixel).
What i have for now is that unfinished function i started:
RGB
bitfox_get_primecolor_direct
(char *FILE_NAME)
{
dword size = bmp_dgets(FILE_NAME, byte);
FILE* fp = fopen(convert(FILE_NAME), "r");
BYTE *PIX_ARRAY = malloc(size-54+1), *PIX_CUR = calloc(sizeof(RGB), sizeof(BYTE));
dword readed, i, l;
RGB color, prime_color;
fseek(fp, 54, SEEK_SET); readed = fread(PIX_ARRAY, 1, size-54, fp);
for(i = 54; i<size-54; i+=3)
{
color = bitfox_pixel_init(PIXEL_ARRAY[i], PIXEL_ARRAY[i+1], PIXEL_ARRAY[i+2);
memmove(PIX_CUR, color, sizeof(RGB));
for(l = 54; l<size-54; l+=3)
{
if (PIX_CUR[2] == PIXEL_ARRAY[l] && PIX_CUR[1] == PIXEL_ARRAY[l+1] &&
PIX_CUR[0] == PIXEL_ARRAY[l+2])
{
}
Note that RGB is a struct containing 3 bytes (R, G and B).
I know thats nothing but.. thats all i have for now.
Is there any way i can finish this?
If you want this done fast throw a stack of RAM at it (if available, of course). You can use a large direct-lookup table with the RGB trio to manufacture a sequence of 24bit indexes into a contiguous array of counters. In partial-pseudo, partial code, something like this:
// create a zero-filled 2^24 array of unsigned counters.
uint32_t *counts = calloc(256*256*256, sizeof(*counts));
uint32_t max_count = 0
// enumerate your buffer of RGB values, three bytes at a time:
unsigned char rgb[3];
while (getNextRGB(src, rgb)) // returns false when no more data.
{
uint32_t idx = (((uint32_t)rgb[0]) << 16) | (((uint32_t)rgb[1]) << 8) | (uint32_t)rgb[2];
if (++counts[idx] > max_count)
max_count = idx;
}
R = (max_count >> 16) & 0xFF;
G = (max_count >> 8) & 0xFF;
B = max_count & 0xFF;
// free when you have no more images to process. for each new
// image you can memset the buffer to zero and reset the max
// for a fresh start.
free(counts);
Thats it. If you can afford to throw a big hulk of memory at this a (it would be 64MB in this case, at 4 bytes per entry at 16.7M entries), then performing this becomes O(N). If you have a succession of images to process you can simply memset() the array back to zeros, clear max_count, and repeat for each additional file. Finally, don't forget to free your memory when finished.
Best of luck.
I need to optimize this function: Any strange way to optimize the for loop? (early break i think can't be possible)
void SeeedGrayOLED::putChar(unsigned char C)
{
if(C < 32 || C > 127) //Ignore non-printable ASCII characters. This can be modified for multilingual font.
{
C=' '; //Space
}
uint8_t k,offset = 0;
char bit1,bit2,c = 0;
for(char i=0;i<16;i++)
{
for(char j=0;j<32;j+=2)
{
if(i>8){
k=i-8;
offset = 1;
}else{
k=i;
}
// Character is constructed two pixel at a time using vertical mode from the default 8x8 font
c=0x00;
bit1=(pgm_read_byte(&hallfetica_normal[C-32][j+offset]) >> (8-k)) & 0x01;
bit2=(pgm_read_byte(&hallfetica_normal[C-32][j+offset]) >> ((8-k)-1)) & 0x01;
// Each bit is changed to a nibble
c|=(bit1)?grayH:0x00;
c|=(bit2)?grayL:0x00;
sendData(c);
}
}
}
I've got a font in the array hallfetica_normal, is an array of array of uint8_t, that maybe compressed or something like that?
This code run on a arduino, ad i've to run a countdown from 500 to 0 with one unit down every 10/20ms.
EDIT
This is the new code after yours indication, thanks all:
I'm looking to organise the font differently to permit less call to pgm_read_byte.. (something like changing the orientation... i wonder)
void SeeedGrayOLED::putChar(unsigned char C)
{
if(C < 32 || C > 127) //Ignore non-printable ASCII characters. This can be modified for multilingual font.
{
C=' '; //Space
}
char c,byte = 0x00;
unsigned char nibble_lookup[] = { 0, grayL, grayH, grayH | grayL };
for(int ii=0;ii<2;ii++){
for(int i=0;i<8;i++)
{
for(int j=0;j<32;j+=2)
{
byte = pgm_read_byte(&hallfetica_normal[C-32][j+ii]);
c = nibble_lookup[(byte >> (8-i)) & 3];
sendData(c);
}
}
}
}
Well, you seem to be reading the same byte twice in a row unnecessarily via pgm_read_byte(&hallfetica_normal[C-32][j+offset]). You could load that once into a local variable.
Additionally, you could avoid the if(i>8){ check per iteration by breaking up the code into two loops; one where i goes from 0 to 8 and another where it goes from 9 to 15. (Although I suspect you really intended >= here, making the loop boundaries 0-7 then 8-15.) That also means things like offset become constant values, which will help.
In an effort to make the inner loop as fast as possible, I'd try to get rid of all branching with a lookup table and see whether that helped.
First, I'd define the lookup table outside the loop:
/* outside the loop */
unsigned char h_lookup[] = { 0, grayH };
unsigned char l_lookup[] = { 0, grayL };
Then inside the loop, since you're testing the least-significant bit, you can use that as an index into the lookup table. If it's clear, then the lookup index will be 0. If it's set, then the lookup index will be 1:
/* inside the loop */
byte = pgm_read_byte(&hallfetica_normal[C-32][j+offset]);
c = h_lookup[((byte >> (8-k)) & 0x01)] |
l_lookup[((byte >> (8-k-1)) & 0x01)]
sendData(c);
Since you're masking and testing 2 adjacent bits, 8-k and 8-k-1, you could list all 4 possibilities in a single lookup table:
/* Outside loop */
unsigned char nibble_lookup[] = { 0, grayL, grayH, grayH | grayL };
And then the lookup becomes dramatically simplified.
/* loop */
byte = pgm_read_byte(&hallfetica_normal[C-32][j+offset]);
c = nibble_lookup[(byte >> (8-k)) & 3];
sendData(c);
The other answer has addressed what to do about the branches in the top part of your inner loop.
My question is mostly about how interpret a typedef enum, but here is the background:
I am using MailCore2, and I am trying to figure out how to read the flags off of an individual email object that I am fetching.
Each MCOIMAPMessage *email that I fetch has a property on it called 'flags.' Flags is of type MCOMessageFlag. When I look up the definition of MCOMessageFlag, I find that it is a typedef enum:
typedef enum {
MCOMessageFlagNone = 0,
/** Seen/Read flag.*/
MCOMessageFlagSeen = 1 << 0,
/** Replied/Answered flag.*/
MCOMessageFlagAnswered = 1 << 1,
/** Flagged/Starred flag.*/
MCOMessageFlagFlagged = 1 << 2,
/** Deleted flag.*/
MCOMessageFlagDeleted = 1 << 3,
/** Draft flag.*/
MCOMessageFlagDraft = 1 << 4,
/** $MDNSent flag.*/
MCOMessageFlagMDNSent = 1 << 5,
/** $Forwarded flag.*/
MCOMessageFlagForwarded = 1 << 6,
/** $SubmitPending flag.*/
MCOMessageFlagSubmitPending = 1 << 7,
/** $Submitted flag.*/
MCOMessageFlagSubmitted = 1 << 8,
} MCOMessageFlag;
Since I do not know how typedef enums really work - particularly this one with the '= 1 << 8' type components, I am a little lost about how to read the emails' flags property.
For example, I have an email message that has both an MCOMessageFlagSeen and an MCOMessageFlagFlagged on the server. I'd like to find out from the email.flags property whether or not the fetched email has one, both or neither of these flags (if possible). However, in the debugger when I print 'email.flags' for an email that has both of the above flags, I get back just the number 5. I don't see how that relates to the typedef enum definitions above.
Ultimately, I want to set a BOOL value based on whether or not the flag is present. In other words, I'd like to do something like:
BOOL wasSeen = email.flags == MCOMessageFlagSeen;
BOOL isFlagged = email.flags == MCOMessageFlagFlagged;
Of course this doesn't work, but this is the idea. Can anyone suggest how I might accomplish this and/or how to understand the typedef enum?
These flags are used as in a bitmask.
This allows to store multiple on/off flags in a single numeric type (let it be an unsigned char or an unsigned int). Basically if a flag is set then its corresponding bit is set too.
For example:
MCOMessageFlagMDNSent = 1 << 5
1<<5 means 1 shifted to the left by 5 bits, so in binary:
00000001 << 5 = 00100000
This works only if no flag overlaps with other flags, which is typically achieved by starting with 1 and shifting it to the left by a different amount for every flag.
Then to check if a flag is set you check if the corresponding bit is set, eg:
if (flags & MCOMessageFlagMDNSent)
result will be true if the bitwise AND result is different from zero, this can happen only if the corresponding bit is set.
You can set a flag easily with OR:
flags |= MCOMessageFlagMDNSent;
or reset it with AND:
flags &= ~MCOMessageFlagMDNSent;
The values of the enum represent the individual bits, so you need bitwise operations to check for flags:
BOOL wasSeen = ( email.flags & MCOMessageFlagSeen ); // check if a bit was set
BTW: You code seems to suggest this is C, not C++. Tagging a question is both is almost always wrong, I suggest you pick the language you are using and remove the other tag.
I saw this line of code today and had no idea what it does.
typedef enum {
SomeOptionKeys = 1 << 0 // ?
} SomeOption;
Some usage or example would be helpful. Thanks!
It looks like it defines an enumerated type that is supposed to contain a set of flags. You'd expect to see more of them defined, like this:
typedef enum {
FirstOption = 1 << 0,
SecondOption = 1 << 1,
ThirdOption = 1 << 2
} SomeOption;
Since they are defined as powers of two, each value corresponds to a single bit in an integer variable. Thus, you can use the bitwise operators to combine them and to test if they are set. This is a common pattern in C code.
You could write code like this that combines them:
SomeOption myOptions = FirstOption | ThirdOption;
And you could check which options are set like this:
if (myOptions & ThirdOption)
{
...
}
The value of SomeOptionKeys is one, this is a useful representation when working with flags:
typedef enum {
flag1 = 1 << 0, // binary 00000000000000000000000000000001
flag2 = 1 << 1, // binary 00000000000000000000000000000010
flag3 = 1 << 2, // binary 00000000000000000000000000000100
flag4 = 1 << 3, // binary 00000000000000000000000000001000
flag5 = 1 << 4, // binary 00000000000000000000000000010000
// ...
} SomeOption;
Whit way each flag has only one bit set, and they could be represented in a bitmap.
Edit:
Although, I have to say, that I might be missing something, but it seems redundent to me to use enums for that. Since you lose any advantage of enums in this configuration, you may as well use #define:
#define flag1 (1<<0)
#define flag2 (1<<1)
#define flag3 (1<<2)
#define flag4 (1<<3)
#define flag5 (1<<4)
It just sets the enum to the value 1. It is probably intended to indicate that the values are to be powers of 2. The next one would maybe be assigned 1 << 1, etc.
<< is the left shift operator. In general, this is used when you want your enums to mask a single bit. In this case, the shift doesn't actually do anything since it's 0, but you might see it pop up in more complex cases.
An example might look like:
typedef enum {
OptionKeyA = 1<<0,
OptionKeyB = 1<<1,
OptionKeyC = 1<<2,
} OptionKeys;
Then if you had some function that took an option key, you could use the enum as a bitmask to check if an option is set.
int ASet( OptionKeys x){
return (x & OptionKeyA);
}
Or if you had a flag bitmap and wanted to set one option:
myflags | OptionKeyB