Related
I’d like to convert a string containing only integers to an array of bytes, but to be stored efficiently (so no “digits[digitIndex] = string[digitIndex - ‘0’;”). I would like them to be stored like any type is stored: having 256 different possibilities per byte, not only 10 as in the previous, faulty example. It also needs to hold a lot of digits (I’m using an 8-bit parameter as the size, so at least 100 digits I believe). Edit: I also do not want to use any libraries whatsoever for personal reasons.
Here’s an example of what it would look like in a function:
int8_t *stringToBigInt(char *input) {
uint8_t digitsBase10 = strlen(input);
uint8_t bytes = ???; //However many bytes to store the result (max 255 bytes in this case)
int8_t *result = malloc(sizeof(void *) + bytes);
... //Code for setting result to input
return result;
}
And here’s an example of a possible input and output:
Edit: This is a short example that fits into 32-bits only for simplicity; an input could be much more than a 32-bit (and possibly 64-bit) integer
Input: “1234567890”
Output: {01001001, 10010110, 00000010, 11010010}
This is a base conversion from base-10 to base-256, so that’s what you should look for as far as algorithms go. For a simplistic implementation, first implement long division by powers of 2 working on strings. Then convert each of the remainders to a byte: these bytes form your output. You’ll want to repeatedly divide the input, and each string of 8 remainder bit remainders forms the base-256 bytes, starting at the least significant digit (one byte is one base-256 digit). Repeated division means that you feed the quotient of the preceding division to the succeeding one, as the dividend.
There are some cool algorithms that can divide base-10 numbers by powers of two, that operate much faster and are simpler than generalized long division. As a hint, let’s take an example: 510. We divide each digit by two, and feed the remainder*5 to the next digit. Let’s drop the fractional part smaller than 0.5: 510 becomes 2*100 + 5*10 + 5. Then 1*100 + 2*10 + 2 dot 5. Then 6*10 + 1. Then 3*10 dot 5, 2*10 + 5, then 1*10 + 2 dot 5, then 6, then 3, then 2 dot 5, then 1, then 0 dot 5.
For 255 we’d get 127.5, 63.5, 15.5, 7.5, 3.5, 1.5, 0.5.
Division by higher factors of two is possible, but requires repeated long additions. E.g. 33 div 4 = 0*10 + 7rem1 + 0 rem 0.75 (ha!). Divisions by two work better since we use the fact that 10=2*5, and base-n notation can be divided by factors of the base easily, without performing long additions: all operations are limited to two adjacent digits, so it’s a linear time process with cost N in number of digits. But for base conversion to base-256 you do repeated division, so the cost is ~0.5N^2. Easy to implement but costly in computations.
There are better algorithms than that, of course. But the above can be implemented concisely - even in the form of reasonably good quality library functions:
First, let's define an array-of-bytes type, and a way to dump it to human-readable hexadecimal output. For convenience, the object is referred to via the pointer to its data, and the implementation detail doesn't figure anywhere in the interface at all. The constructor new_Bytes zero-initializes the array. There is also a method that treats the array as if it was an array of bits, ordered lest-endian (LSB first), and sets (turns on) a given bit.
// https://github.com/KubaO/stackoverflown/tree/master/questions/decimal-to-binary-54422895
#include <assert.h>
#include <inttypes.h>
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
// Bytes Class Interface
typedef uint8_t *Bytes;
typedef const uint8_t *cBytes;
Bytes new_Bytes(size_t size);
size_t Bytes_size(cBytes bytes);
void Bytes_truncate(Bytes bytes, size_t new_size);
void free_Bytes(cBytes bytes);
char *Bytes_to_hex(cBytes bytes);
static inline void Bytes_set_bit(Bytes const bytes, size_t const bit_num) {
bytes[bit_num / 8] |= 1 << (bit_num % 8);
}
Then, the division-by-2 is performed in-place, and the flags provide additional information needed for base conversion - especially the remainder. The conversion from base 10 to base 256 uses the division and returns a new Bytes array.
// Division and Base Conversion Interface
typedef enum {
REMAINDER = 1, /* there is a non-zero remainder */
ZERO = 2, /* the quotient is zero or null */
NULL_DECIMAL = 4, /* the dividend is null or empty */
NON_DECIMALS = 8, /* division was terminated on non-decimal characters */
LEADING_ZERO_COUNT = 16, /* count of leading zeroes in the quotient */
LEADING_ZERO_COUNT_MASK = ~(LEADING_ZERO_COUNT - 1),
CLR_CARRY_MASK = ~REMAINDER,
CLR_ZERO_MASK = ~ZERO,
} DivFlags;
DivFlags divide_by_2(char *decimal);
Bytes base_10_to_256(const char *decimal);
The division operates on the decimal representation, in order from most-significant to least-significant digit. Each digit is merged with the remainder from the prior digit's division, and then is divided by 2. The remainder is carried between digit divisions. After division of the least significant digit, the remainder is output in the flags.
The flags are mostly self-explanatory, but LEADING_ZERO_COUNT isn't quite - and thus the access to it is implemented via accessor functions. LEADING_ZERO_COUNT is the unit of the count of leading zeroes. As the division steps though the decimal representation, it will count the leading zeroes, multiply them by this unit, and merge it with the flags. To extract the count, the flags are divided by the unit.
// Division and Base Conversion Implementation
static inline int leading_zero_count(DivFlags const flags) {
return (flags & LEADING_ZERO_COUNT_MASK) / LEADING_ZERO_COUNT;
}
static inline void saturated_inc_leading_zero_count(DivFlags *flags) {
if ((*flags & LEADING_ZERO_COUNT_MASK) != LEADING_ZERO_COUNT_MASK)
*flags += LEADING_ZERO_COUNT;
}
DivFlags divide_by_2(char *decimal) {
DivFlags flags = ZERO;
if (!decimal) return flags | NULL_DECIMAL;
char c;
while ((c = *decimal)) {
if (c < '0' || c > '9') return flags | NON_DECIMALS;
c = c - '0' + ((flags & REMAINDER) ? 10 : 0);
if (c & 1)
flags |= REMAINDER;
else
flags &= CLR_CARRY_MASK;
c >>= 1;
assert(c >= 0 && c <= 9);
if (c)
flags &= CLR_ZERO_MASK;
else if (flags & ZERO)
saturated_inc_leading_zero_count(&flags);
*decimal++ = c + '0';
}
return flags;
}
Then, the base conversion performs repeated division by 2, and shifts the remainder bits into the byte array, as follows:
First, the base conversion takes a copy of the decimal representation, and allocates the output byte array of the appropriate size.
static void base_10_to_256_impl(Bytes const bytes, char *decimal);
Bytes base_10_to_256(const char *const decimal) {
assert(decimal);
size_t const dec_len = strlen(decimal);
char *const dec_buf = malloc(dec_len + 1);
if (!dec_buf) return NULL;
memcpy(dec_buf, decimal, dec_len + 1);
size_t const BASE_RATIO_NUM = 416, /* ceil(log(10)/log(256)*1000) */
BASE_RATIO_DENOM = 1000;
assert(dec_len <= (SIZE_MAX / BASE_RATIO_NUM));
size_t const len = (size_t)(dec_len * BASE_RATIO_NUM / BASE_RATIO_DENOM) + 1;
Bytes const bytes = new_Bytes(len); // little-endian
if (bytes) base_10_to_256_impl(bytes, dec_buf);
free(dec_buf);
return bytes;
}
Then, in the "meat" of the implementation, the function iterates the output bits, repeatedly dividing the decimal representation by 2, and sets each bit with the value of the remainder bit.
static void base_10_to_256_impl(Bytes const bytes, char *decimal) {
size_t const len = Bytes_size(bytes);
for (size_t bit_num = 0;; bit_num++) {
DivFlags const flags = divide_by_2(decimal);
assert(!(flags & NULL_DECIMAL));
decimal += leading_zero_count(flags);
if (flags & ZERO && !(flags & REMAINDER)) {
size_t const new_len = ((bit_num + 7) / 8);
Bytes_truncate(bytes, new_len);
break;
}
// here, there are still non-zero bits - in the dec[imal] and/or in the carry
assert((bit_num / 8) < len);
if (flags & REMAINDER) Bytes_set_bit(bytes, bit_num);
}
}
We can now add some tests:
// Tests
void check_bytes(const char *const decimal, const char *const bytes_expected,
size_t const bytes_len, const char *const hex_expected) {
cBytes const bytes = base_10_to_256(decimal);
assert(bytes && Bytes_size(bytes) == bytes_len);
assert(memcmp(bytes, bytes_expected, bytes_len) == 0);
char *const hex = Bytes_to_hex(bytes);
assert(hex && strcmp(hex, hex_expected) == 0);
printf("%s\n", hex);
free(hex);
free_Bytes(bytes);
}
int main() {
check_bytes("4294967297" /* 2^32+1 */, "\1\0\0\0\1", 5, "01 00000001");
check_bytes("4294967296" /* 2^32 */, "\0\0\0\0\1", 5, "01 00000000");
check_bytes("4294967295" /* 2^32-1 */, "\xFF\xFF\xFF\xFF", 4, "FFFFFFFF");
check_bytes("16777217" /* 2^24+1 */, "\1\0\0\1", 4, "01000001");
check_bytes("16777216" /* 2^24 */, "\0\0\0\1", 4, "01000000");
check_bytes("16777215" /* 2^24-1 */, "\xFF\xFF\xFF", 3, "FFFFFF");
check_bytes("256", "\0\1", 2, "0100");
check_bytes("255", "\xFF", 1, "FF");
check_bytes("254", "\xFE", 1, "FE");
check_bytes("253", "\xFD", 1, "FD");
check_bytes("3", "\3", 1, "03");
check_bytes("2", "\2", 1, "02");
check_bytes("1", "\1", 1, "01");
check_bytes("0", "\0", 1, "00");
}
The implementation of the Bytes class concludes the example:
// Bytes Implementation
struct BytesImpl {
size_t size;
uint8_t data[1];
};
static const size_t Bytes_header_size = offsetof(struct BytesImpl, data);
_Static_assert(offsetof(struct BytesImpl, data) == sizeof(size_t),
"unexpected layout of struct BytesImpl");
Bytes new_Bytes(size_t size) {
assert(size <= SIZE_MAX - Bytes_header_size);
if (!size) size++;
struct BytesImpl *const impl = calloc(Bytes_header_size + size, 1);
if (!impl) return NULL;
impl->size = size;
return &impl->data[0];
}
static const struct BytesImpl *Bytes_get_const_impl_(cBytes const bytes) {
return (const struct BytesImpl *)(const void *)((const char *)bytes -
Bytes_header_size);
}
static struct BytesImpl *Bytes_get_impl_(Bytes const bytes) {
return (struct BytesImpl *)(void *)((char *)bytes - Bytes_header_size);
}
size_t Bytes_size(cBytes const bytes) { return Bytes_get_const_impl_(bytes)->size; }
void Bytes_truncate(Bytes const bytes, size_t new_size) {
size_t *const size = &Bytes_get_impl_(bytes)->size;
if (!new_size) {
new_size++; // we always leave one byte in the array
bytes[0] = 0;
}
assert(*size);
if (*size <= new_size) return;
*size = new_size;
}
void free_Bytes(cBytes const bytes) {
if (bytes) free((void *)(intptr_t)(const void *)Bytes_get_const_impl_(bytes));
}
char *Bytes_to_hex(cBytes const bytes) {
size_t n = Bytes_size(bytes);
size_t spaces = (n - 1) / 4;
char *const out = malloc(n * 2 + spaces + 1);
if (out)
for (char *o = out; n;) {
uint8_t const c = bytes[n - 1];
snprintf(o, 3, "%02" PRIX8, c);
o += 2;
n--;
if (n && n % 4 == 0) {
assert(spaces);
*o++ = ' ';
spaces--;
}
}
return out;
}
Lets assume I have an input array like below
int input_arr[100] = {10,20,1255,1200,50,55,1,5,6,1000};
Here to store each elements of array it took 32 bits even though value of array elements is very small i.e 1255 is the maximum elements in array and to store that I need only 11 bit that means in 11 bit I can fit all other elements of array.
So my task to compress 32-bit elements of array into 11-bit array elements ? Expected compressed array looks like
int output_arr[] = {00000001010 00000010100 .... 10011111111 ... }
| | |
11 bits(1) 11 bits(2) 11 bits( 1255)
To do the above task what I did is here
find the maximum elements in given array
find the bits required to store maximum elements(previous step)
find bytes required to store no of bits for e.g to store 11 bits I need equivalent 2 bytes(in below code new_size contains this). Here is I need your help. Here is the memory wastage as told by my manager because to store 11 bits my new_size is 2 bytes i.e 5 bits are still extra or wastage. How can I avoid this.
Here is what I tried
int my_pow(int input_num,int p) {
int temp = 1;
for(int iter = 0;iter < p; iter++) {
temp = temp * input_num;
}
return temp;
}
int main() {
#if 0
int input_array[53069] = {1,2,2,3,4,1,2,4,6,1255,1,2,5,1233};
#endif
int input_array[] = {1,2,3,4,6,1255,1,2,5,1233};
int max = input_array[0], ele = sizeof(input_array)/sizeof(input_array[0]);
/* finding max elements in a array */
for(int i = 0;i < ele; i++) {
if(input_array[i] > max) {
max = input_array[i];
}
}
/* finding no of bits required to store highest elements of array */
int bit_required = 0;
while(1) {
if(max < my_pow(2,bit_required))
break;
bit_required+=1;
}
/* when above loop fails bit_required is nothing
but no of bit required to store the highest element of array */
/* finding size of new/compressed array */
int new_size = 0;
if(bit_required % 8 == 0) {
new_size = bit_required/8;
}
else {
new_size = (bit_required/8) + 1;
}
/* construct the new array again */
typedef struct array_task {
unsigned char new_array[new_size];/* in each cmp_arr, can store new_size char
now for each B[] I'm not using 32 bits , its new_size bits */
}cmp_arr;/* creating new array of ele elements */
cmp_arr cmpressed[ele];
/* store elements of input_array[] into output_array[] */
for(int row = 0 ; row < ele ;row++) {
for(int col = bit_required - 1; col >= 0; col-- ) {
cmpressed[row].new_array[col] = ((input_array[row] >> col & 1) + 48) ;
printf("%d",(cmpressed[row].new_array[col]) - 48);
}
printf("\n");
}
#if 0
printf("Size of A before %d\n",sizeof(input_array)); /* 40 bytes */
printf("size of compressed array %d\n",sizeof(cmp_arr));/* same task, it perform in 2 bytes,
each elements won't take 32 bits */
#endif
return 0;
}
Is there any other way to do the same task efficiently ? All suggestion are most welcome ?
To put values shifted by 11 bits instead of 8, 16 or 32 will require manipulations with bits. You will basically have to emulate an array of bits in an array of (say 32 bits) integers. In this case if a value is stored at a bit offset X it will be (possibly) stored in your array somewhere on indexes X/32 and X/32+1 (if it is crossing border of 32 bits). Each time when you have to set a value into the array you have to load those two values and "place" your number there. The implementation is a bit technical, try the following code:
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#define MASK32 ((uint64_t)0xffffffff)
void setValue(uint32_t *a, int bits, int i, int n) {
int bitoffset = i * bits;
int index = bitoffset / 32;
int shift = bitoffset % 32;
uint64_t maskbits = (~(uint64_t)0) >> (64-bits);
uint64_t val = ((uint64_t)a[index+1]<<32) + a[index];
val = val & ~(maskbits << shift) | ((n & maskbits) << shift);
a[index] = (val & MASK32);
a[index+1] = (val >> 32) & MASK32;
}
int getValue(const uint32_t *a, int bits, int i) {
int bitoffset = i * bits;
int index = bitoffset / 32;
int shift = bitoffset % 32;
uint64_t maskbits = (~(uint64_t)0) >> (64-bits);
int val = ((((uint64_t)a[index+1]<<32) + a[index]) >> shift) & maskbits;
return(val);
}
int input_arr[100] = {10,20,1255,1200,50,55,1,5,6,1000};
int main() {
int i, j;
uint32_t a[100*11/32+2];
for(i=0; i<100; i++) setValue(a,11,i,input_arr[i]);
for(j=0; j<100; j++) printf("a[%d/11] == %d\n", j, getValue(a,11,j));
}
Another approach that I find "interesting" is allocating an array of chars and doing a cast to an type that fits the maximum value. Something like this:
NumBytesMaxValue = ...;
void* pointers = malloc(NumBytesMaxValue * NumValues);
if (NumBytesMaxValue == 1)
cast_pointer_to_char_and_fill_it();
else if (NumBytesMaxValue == 2)
cast_pointer_to_short_and_fill_it();
...
Data compression is a vast subject, an active area of research... compressing your data can be done in so many different ways as to make it off topic.
Finding the smallest type for the array can however be done by a utility program or a preliminary phase:
#include <limits.h>
#include <stdio.h>
int main() {
int input_array[] = { 1, 2, 2, 3, 4, 1, 2, 4, 6, 1255, 1, 2, 5, 1233 };
size_t i, count = sizeof(input_array) / sizeof(input_array[0]);
int min, max;
int nc = 0;
min = max = input_array[0];
for (i = 1; i < count; i++) {
if (min > input_array[i]) min = intput_array[i];
if (max < input_array[i]) max = intput_array[i];
}
printf("min value is %d, max value is %d\n", min, max);
if (min >= SCHAR_MIN && max <= SCHAR_MAX)
nc += printf("type signed char is appropriate\n");
if (min >= 0 && max <= UCHAR_MAX)
nc += printf("type unsigned char is appropriate\n");
if (min >= SHRT_MIN && max <= SHRT_MAX)
nc += printf("type short is appropriate\n");
if (min >= 0 && max <= USHRT_MAX)
nc += printf("type unsigned short is appropriate\n");
if (nc == 0)
printf("no type smaller than int is appropriate\n");
return 0;
}
You can use the same approach for a set of numbers with values unknown at compile time with these steps:
start with an allocated array of a small type such as signed char.
read the next value: if it fits in the current type, add it to the array and continue.
if not, allocate an array of a larger type such as short, copy the values parsed so far into it, free the previous array, store the new value and continue.
if the new value does not fit in a short, use a larger type such as int.
you could write code for even larger types such as long and long long, but you need specific code for each type.
at the end of the read phase, you have an array of the smallest type that handles all the values in the dataset. Handle this array with code for its specific type. This means you have to duplicate the processing code for each type, which can be tricky.
I am trying to implement a 60kHz bandpass filter on the STM32F407 microcontroller and I'm having some issues. I have generated the filter with the help of MATLABs fdatool and then simulated it in MATLAB as well. The following MATLAB script simlates it.
% FIR Window Bandpass filter designed using the FIR1 function.
% All frequency values are in Hz.
Fs = 5250000; % Sampling Frequency
N = 1800; % Order
Fc1 = 59950; % First Cutoff Frequency
Fc2 = 60050; % Second Cutoff Frequency
flag = 'scale'; % Sampling Flag
% Create the window vector for the design algorithm.
win = hamming(N+1);
% Calculate the coefficients using the FIR1 function.
b = fir1(N, [Fc1 Fc2]/(Fs/2), 'bandpass', win, flag);
Hd = dfilt.dffir(b);
%----------------------------------------------------------
%----------------------------------------------------------
T = 1 / Fs; % sample time
L = 4500; % Length of signal
t = (0:L-1)*T; % Time vector
% Animate the passband frequency span
for f=55500:50:63500
signal = sin(2*pi*f*t);
plot(filter(Hd, signal));
axis([0 L -1 1]);
str=sprintf('Signal frequency (Hz) %d', f);
title(str);
drawnow;
end
pause;
close all;
signal = sin(2*pi*50000*t) + sin(2*pi*60000*t) + sin(2*pi*78000*t);
signal = signal / 3;
signal = signal(1:1:4500);
filterInput = signal;
filterOutput = filter(Hd,signal);
subplot(2,1,1);
plot(filterInput);
axis([0 4500 -1 1]);
subplot(2,1,2);
plot(filterOutput)
axis([0 4500 -1 1]);
pause;
close all;
From the fdatool I extract the filter co-efficents to 16-bit unsigned integers in q15 format, this because of the 12-bit ADC that I'm using. The filter co-efficents header that is generated by MATLAB is here and the resulting plot of the co-efficents can be seen in the following picture
Below is the code for the filter implementation which obviously isn't working and I don't really know what I can do differently, I've looked at some examples online Example 1 and Example 2
#include "fdacoefs.h"
#define FILTER_SAMPLES 4500
#define BLOCK_SIZE 900
static uint16_t firInput[FILTER_SAMPLES];
static uint16_t firOutput[FILTER_SAMPLES];
static uint16_t firState[NUM_TAPS + BLOCK_SIZE - 1];
uint16_t util_calculate_filter(uint16_t *buffer, uint32_t len)
{
uint16_t i;
uint16_t max;
uint16_t min;
uint32_t index;
// Create filter instance
arm_fir_instance_q15 instance;
// Ensure that the buffer length isn't longer than the sample size
if (len > FILTER_SAMPLES)
len = FILTER_SAMPLES;
for (i = 0; i < len ; i++)
{
firInput[i] = buffer[i];
}
// Call Initialization function for the filter
arm_fir_init_q15(&instance, NUM_TAPS, &firCoeffs, &firState, BLOCK_SIZE);
// Call the FIR process function, num of blocks to process = (FILTER_SAMPLES / BLOCK_SIZE)
for (i = 0; i < (FILTER_SAMPLES / BLOCK_SIZE); i++) //
{
// BLOCK_SIZE = samples to process per call
arm_fir_q15(&instance, &firInput[i * BLOCK_SIZE], &firOutput[i * BLOCK_SIZE], BLOCK_SIZE);
}
arm_max_q15(&firOutput, len, &max, &index);
arm_min_q15(&firOutput, len, &min, &index);
// Convert output back to uint16 for plotting
for (i = 0; i < (len); i++)
{
buffer[i] = (uint16_t)(firOutput[i] - 30967);
}
return (uint16_t)((max+min));
}
The ADC is sampling at 5.25 MSPS and it is sampling a 60kHz signal 4500 times and here you can see the Input to the filter and then the Output of the filter which is pretty weird..
Is there anything obvious that I've missed? Because I'm completely lost and any pointers and tips are helpful!
As Lundin pointed out I changed it to work with 32 bit integers instead and that actually solved my problem. Ofcourse I generated new filter co-efficents with MATLABS fdatool as signed 32 bit integers instead.
static signed int firInput[FILTER_SAMPLES];
static signed int firOutput[FILTER_SAMPLES];
static signed int firState[NUM_TAPS + BLOCK_SIZE -1];
uint16_t util_calculate_filter(uint16_t *buffer, uint32_t len)
{
uint16_t i;
int power;
uint32_t index;
// Create filter instance
arm_fir_instance_q31 instance;
// Ensure that the buffer length isn't longer than the sample size
if (len > FILTER_SAMPLES)
len = FILTER_SAMPLES;
for (i = 0; i < len ; i++)
{
firInput[i] = (int)buffer[i];
}
// Call Initialization function for the filter
arm_fir_init_q31(&instance, NUM_TAPS, &firCoeffs, &firState, BLOCK_SIZE);
// Call the FIR process function, num of blocks to process = (FILTER_SAMPLES / BLOCK_SIZE)
for (i = 0; i < (FILTER_SAMPLES / BLOCK_SIZE); i++) //
{
// BLOCK_SIZE = samples to process per call
//arm_fir_q31(&instance, &firInput[i * BLOCK_SIZE], &firOutput[i * BLOCK_SIZE], BLOCK_SIZE);
arm_fir_q31(&instance, &firInput[i * BLOCK_SIZE], &firOutput[i * BLOCK_SIZE], BLOCK_SIZE);
}
arm_power_q31(&firOutput, len, &power);
// Convert output back to uint16 for plotting
for (i = 0; i < (len); i++)
{
buffer[i] = (uint16_t)(firOutput[i] - 63500);
}
return (uint16_t)((power/10));
}
I need generate random 64-bit unsigned integers using C. I mean, the range should be 0 to 18446744073709551615. RAND_MAX is 1073741823.
I found some solutions in the links which might be possible duplicates but the answers mostly concatenates some rand() results or making some incremental arithmetic operations. So results are always 18 digits or 20 digits. I also want outcomes like 5, 11, 33387, not just 3771778641802345472.
By the way, I really don't have so much experience with the C but any approach, code samples and idea could be beneficial.
Concerning "So results are always 18 digits or 20 digits."
See #Thomas comment. If you generate random numbers long enough, code will create ones like 5, 11 and 33387. If code generates 1,000,000,000 numbers/second, it may take a year as very small numbers < 100,000 are so rare amongst all 64-bit numbers.
rand() simple returns random bits. A simplistic method pulls 1 bit at a time
uint64_t rand_uint64_slow(void) {
uint64_t r = 0;
for (int i=0; i<64; i++) {
r = r*2 + rand()%2;
}
return r;
}
Assuming RAND_MAX is some power of 2 - 1 as in OP's case 1073741823 == 0x3FFFFFFF, take advantage that 30 at least 15 bits are generated each time. The following code will call rand() 5 3 times - a tad wasteful. Instead bits shifted out could be saved for the next random number, but that brings in other issues. Leave that for another day.
uint64_t rand_uint64(void) {
uint64_t r = 0;
for (int i=0; i<64; i += 15 /*30*/) {
r = r*((uint64_t)RAND_MAX + 1) + rand();
}
return r;
}
A portable loop count method avoids the 15 /*30*/ - But see 2020 edit below.
#if RAND_MAX/256 >= 0xFFFFFFFFFFFFFF
#define LOOP_COUNT 1
#elif RAND_MAX/256 >= 0xFFFFFF
#define LOOP_COUNT 2
#elif RAND_MAX/256 >= 0x3FFFF
#define LOOP_COUNT 3
#elif RAND_MAX/256 >= 0x1FF
#define LOOP_COUNT 4
#else
#define LOOP_COUNT 5
#endif
uint64_t rand_uint64(void) {
uint64_t r = 0;
for (int i=LOOP_COUNT; i > 0; i--) {
r = r*(RAND_MAX + (uint64_t)1) + rand();
}
return r;
}
The autocorrelation effects commented here are caused by a weak rand(). C does not specify a particular method of random number generation. The above relies on rand() - or whatever base random function employed - being good.
If rand() is sub-par, then code should use other generators. Yet one can still use this approach to build up larger random numbers.
[Edit 2020]
Hallvard B. Furuseth provides as nice way to determine the number of bits in RAND_MAX when it is a Mersenne Number - a power of 2 minus 1.
#define IMAX_BITS(m) ((m)/((m)%255+1) / 255%255*8 + 7-86/((m)%255+12))
#define RAND_MAX_WIDTH IMAX_BITS(RAND_MAX)
_Static_assert((RAND_MAX & (RAND_MAX + 1u)) == 0, "RAND_MAX not a Mersenne number");
uint64_t rand64(void) {
uint64_t r = 0;
for (int i = 0; i < 64; i += RAND_MAX_WIDTH) {
r <<= RAND_MAX_WIDTH;
r ^= (unsigned) rand();
}
return r;
}
If you don't need cryptographically secure pseudo random numbers, I would suggest using MT19937-64. It is a 64 bit version of Mersenne Twister PRNG.
Please, do not combine rand() outputs and do not build upon other tricks. Use existing implementation:
http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/emt64.html
Iff you have a sufficiently good source of random bytes (like, say, /dev/random or /dev/urandom on a linux machine), you can simply consume 8 bytes from that source and concatenate them. If they are independent and have a linear distribution, you're set.
If you don't, you MAY get away by doing the same, but there is likely to be some artefacts in your pseudo-random generator that gives a toe-hold for all sorts of hi-jinx.
Example code assuming we have an open binary FILE *source:
/* Implementation #1, slightly more elegant than looping yourself */
uint64_t 64bitrandom()
{
uint64_t rv;
size_t count;
do {
count = fread(&rv, sizeof(rv), 1, source);
} while (count != 1);
return rv;
}
/* Implementation #2 */
uint64_t 64bitrandom()
{
uint64_t rv = 0;
int c;
for (i=0; i < sizeof(rv); i++) {
do {
c = fgetc(source)
} while (c < 0);
rv = (rv << 8) | (c & 0xff);
}
return rv;
}
If you replace "read random bytes from a randomness device" with "get bytes from a function call", all you have to do is to adjust the shifts in method #2.
You're vastly more likely to get a "number with many digits" than one with "small number of digits" (of all the numbers between 0 and 2 ** 64, roughly 95% have 19 or more decimal digits, so really that is what you will mostly get.
If you are willing to use a repetitive pseudo random sequence and you can deal with a bunch of values that will never happen (like even numbers? ... don't use just the low bits), an LCG or MCG are simple solutions. Wikipedia: Linear congruential generator can get you started (there are several more types including the commonly used Wikipedia: Mersenne Twister). And this site can generate a couple prime numbers for the modulus and the multiplier below. (caveat: this sequence will be guessable and thus it is NOT secure)
#include <stdio.h>
#include <stdint.h>
uint64_t
mcg64(void)
{
static uint64_t i = 1;
return (i = (164603309694725029ull * i) % 14738995463583502973ull);
}
int
main(int ac, char * av[])
{
for (int i = 0; i < 10; i++)
printf("%016p\n", mcg64());
}
I have tried this code here and it seems to work fine there.
#include <time.h>
#include <stdlib.h>
#include <math.h>
int main(){
srand(time(NULL));
int a = rand();
int b = rand();
int c = rand();
int d = rand();
long e = (long)a*b;
e = abs(e);
long f = (long)c*d;
f = abs(f);
long long answer = (long long)e*f;
printf("value %lld",answer);
return 0;
}
I ran a few iterations and i get the following outputs :
value 1869044101095834648
value 2104046041914393000
value 1587782446298476296
value 604955295827516250
value 41152208336759610
value 57792837533816000
If you have 32 or 16-bit random value - generate 2 or 4 randoms and combine them to one 64-bit with << and |.
uint64_t rand_uint64(void) {
// Assuming RAND_MAX is 2^31.
uint64_t r = rand();
r = r<<30 | rand();
r = r<<30 | rand();
return r;
}
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <time.h>
unsigned long long int randomize(unsigned long long int uint_64);
int main(void)
{
srand(time(0));
unsigned long long int random_number = randomize(18446744073709551615);
printf("%llu\n",random_number);
random_number = randomize(123);
printf("%llu\n",random_number);
return 0;
}
unsigned long long int randomize(unsigned long long int uint_64)
{
char buffer[100] , data[100] , tmp[2];
//convert llu to string,store in buffer
sprintf(buffer, "%llu", uint_64);
//store buffer length
size_t len = strlen(buffer);
//x : store converted char to int, rand_num : random number , index of data array
int x , rand_num , index = 0;
//condition that prevents the program from generating number that is bigger input value
bool Condition = 0;
//iterate over buffer array
for( int n = 0 ; n < len ; n++ )
{
//store the first character of buffer
tmp[0] = buffer[n];
tmp[1] = '\0';
//convert it to integer,store in x
x = atoi(tmp);
if( n == 0 )
{
//if first iteration,rand_num must be less than or equal to x
rand_num = rand() % ( x + 1 );
//if generated random number does not equal to x,condition is true
if( rand_num != x )
Condition = 1;
//convert character that corrosponds to integer to integer and store it in data array;increment index
data[index] = rand_num + '0';
index++;
}
//if not first iteration,do the following
else
{
if( Condition )
{
rand_num = rand() % ( 10 );
data[index] = rand_num + '0';
index++;
}
else
{
rand_num = rand() % ( x + 1 );
if( rand_num != x )
Condition = 1;
data[index] = rand_num + '0';
index++;
}
}
}
data[index] = '\0';
char *ptr ;
//convert the data array to unsigned long long int
unsigned long long int ret = _strtoui64(data,&ptr,10);
return ret;
}
I've just started learning C and I'm having some problems with some code I want to write.
Basically I have this struct that is a bit array, with the number of bits in the array, and a pointer to a buffer of chars, that stores the bits.
My strategy for rotating the bit array is simply taking the number of rotations (mod the length to avoid full rotations) and using a simple reversal algorithm to rotate the array.
EDIT:
However, my problem is that I want to rotate the bits in the actual buffer.
I also want to be able to rotate a subsequence of bits within the entire bit array. So for 1101101, I might want to rotate (0-indexed from the left) the subsequence starting at index 2 and ending at index 5. I'm not entirely sure how to use my char buffer to do this.
Thanks for the help!
struct arrayBits{
size_t numBits;
char *buf;
}
The buf array holds 8-bit integers, not bools as I previously mentioned.
The way that I can access and set an individual bit is just by indexing into the byte that holds the bit I want (so for an array ab, ab->buf[index_of_desired_bit/8] and then performing some bitwise operations on it to change the value, for performance reasons.
EDIT: Thanks to everyone for all the suggestions. I've looked at all of them and I believe I understand the code better. Here's the code I ended up writing, however, I think there are some problems with it.
While it passes some of my basic test cases, it seems to run a little too fast on an bitarray of size 98775 bits, randomly filled. By this I mean, is there some case in which my code just outright fails and crashes? The test cases do three rotations, in a row, on the full 98775-bit array. One rotation of -98775/4 (<--this is a size_t, so wrap around?), one rotation of 98775/4, and then a final rotation of 98775/2.
Is there something I'm missing or some problem I'm not seeing?
/*Reverse a bit array*/
/*v1.1: basic bit reversal w/o temp variable*/
static void arrayReversal(bitarray_t *ba, size_t begin, size_t end){
while(begin < end)
{
bitarray_set(ba, begin, (bitarray_get(ba, begin) ^ bitarray_get(ba, end))); /*x = x ^ y*/
bitarray_set(ba, end, (bitarray_get(ba, begin) ^ bitarray_get(ba, end))); /*y = x ^ y*/
bitarray_set(ba, begin, (bitarray_get(ba, begin) ^ bitarray_get(ba, end))); /*x = x ^ y*/
begin++;
end--;
}
}
/*Main Rotation Routine*/
void bitarray_rotate(bitarray_t *ba, size_t bit_off, size_t bit_len, ssize_t bit_right_amount) {
assert(bit_off + bit_len <= ba->bit_sz);
assert(bit_off + bit_len > 0);
if(bit_off + bit_len > ba->bit_sz || bit_off + bit_len < 0)
{
printf("\nError: Indices out of bounds\n");
return;
}
/*Find index to split bitarray on*/
if(bit_len == 0) return; //Rotate only 1 bit i.e. no rotation
size_t reversal_index;
reversal_index = modulo(-bit_right_amount, bit_len);
if(reversal_index == 0) return; //No rotation to do
/*3 bit string reversals*/
assert(reversal_index - 1 + bit_off < ba->bit_sz);
/* Reverse A*/
arrayReversal(ba, bit_off, reversal_index - 1 + bit_off);
assert(reversal_index + bit_off < ba->bit_sz);
/*Reverse B*/
arrayReversal(ba, reversal_index + bit_off, (bit_off + bit_len - 1));
/*Reverse ArBr*/
arrayReversal(ba, bit_off, (bit_off + bit_len -1));
}
Well the easy way to start is to consider how to rotate the bits in a single value. Let's say that you have x, which is an N-bit value and you want to rotate it by k places. (I'm only going to look at rotating upwards/left, it is easy to convert to downwards/right). The first thing to observe is that if k=N then x is unchanged. So before rotating we want to reduce k modulo N to throw away complete rotations.
Next we should observe that during the rotation the k upper-bits will move to the bottom of the value, and the lower N-k bits will move up k places. This is the same as saying that the top k-bits move down N-k places. The reason that we phrase it this way is that C has shift operators, but not rotation.
In psuedo-C we can say:
#define N sizeof(type)*8
type rotate(type x, int k) {
type lower = x & ((1 << (N-k)) - 1);
type upper = x >> (N-k) & ((1 <<k)-1);
return upper | lower;
}
This takes care of the simple atomic case, simply replace type with char or int as appropriate. If type is unsigned then the mask on the value of upper is unnecessary.
The next thing to consider is rotating in an array of values. If you think of the above code as glueing together two halves of a value then for the more complicated case we need to glue together upper and lower parts from different places in the array. If k is small then these places are adjacent in the array, but when k>N we are rotating through more than one intermediate word.
In particular if we are rotating up k places then we are moving bits from k/N words away in the array, and the N bits can span floor(k/N) and ceil(k/N) locations away in the array. Ok, so now we're ready to put it all together. For each word in the array the new upper N-(k mod N) bits will be the lower bits of floor(k/N) words away, and the new lower (k mod N) bits will be the upper bits of ceil(k/N) words away.
In the same psuedo-C (i.e replace type with what you are using) we can say:
#define N sizeof(type)*8
#define ARR_SIZE ...
type rotate(type *x, int k,type *out) {
int r = k % N;
int upperOff = k/N;
int lowerOff = (k+N-1)/N;
for(int i=0; i<ARR_SIZE; i++) {
int lowerPos = (i + ARR_SIZE - lowerOff) % ARR_SIZE
int upperPos = (i + ARR_SIZE - upperOff) % ARR_SIZE
type lower = x[lowerPos] & ((1 << (N-k)) - 1)
type upper = x[upperPos] >> (N-k) & ((1 <<k)-1)
out[i] = upper | lower;
}
}
Anyway, that's a lot more than I was intending to write so I'll quit now. It should be easy enough to convert this to a form that works inplace on a single array, but you'll probably want to fix the types and the range of k first in order to bound the temporary storage.
If you have any more problems in this area then one place to look is bitmap sprite graphics. For example this rotation problem was used to implement scrolling many, many moons ago in 8-bit games.
I would suggest a pointer/offset to a starting point of a bit in the buffer instead of rotating. Feel free to overload any operator that might be useful, operator[] comes to mind.
A rotate(n) would simply be a offset+=n operation. But I find the purpose of your comment about -"However, my problem is that I want to rotate the actual buffer" confusing.
You dont need an extra buffer for rotate (only for output).
You should implement a function for one rotate and loop this, eg: (right-shift variation)
char *itoa2(char *s,size_t i)
{
*s=0;
do {
memmove(s+1,s,strlen(s)+1);
*s='0'+(i&1);
} while( i>>=1 );
return s;
}
size_t bitrotateOne(size_t i)
{
return i>>1 | (i&1) << (sizeof i<<3)-1;
}
...
size_t i=12,num=17;
char b[129];
while( num-- )
{
i = bitrotateOne(i);
puts( itoa2(b,i) );
}
Since your criteria is so complex, I think the easiest way to do it would be to step through each bit and set where it would be in your new array. You could speed it up for some operations by copying a whole character if it is outside the shifted bits, but I can't think of how to reliably do shifting taking into account all the variables because the start and end of the shifted sequence can be in the middle of bytes and so can the end of the entire bits. The key is to get the new bit position for a bit in the old array:
j = (i < startBit || i >= startBit + length) ? i :
((i - startBit + shiftRightCount) % length) + startBit;
Code:
#include "stdafx.h"
#include <stdlib.h>
#include <string.h>
typedef struct {
size_t numBits;
unsigned char *buf;
} ARRAYBITS;
// format is big endian, shiftint left 8 bits will shift all bytes to a lower index
ARRAYBITS rotateBits(ARRAYBITS *pOriginalBits, int startBit, int length, int shiftRightCount);
void setBit(unsigned char *buf, int bit, bool isSet);
bool checkBit(unsigned char *buf, int bit);
ARRAYBITS fromString(char *onesAndZeros);
char *toString(ARRAYBITS *pBits);
int _tmain(int argc, _TCHAR* argv[])
{
char input[1024];
ARRAYBITS bits = fromString("11110000110010101110"); // 20 bits
ARRAYBITS bitsA = rotateBits(&bits, 0, bits.numBits, 1);
ARRAYBITS bitsB = rotateBits(&bits, 0, bits.numBits, -1);
ARRAYBITS bitsC = rotateBits(&bits, 6, 8, 4);
ARRAYBITS bitsD = rotateBits(&bits, 6, 8, -2);
ARRAYBITS bitsE = rotateBits(&bits, 6, 8, 31);
ARRAYBITS bitsF = rotateBits(&bits, 6, 8, -31);
printf("Starting : %s\n", toString(&bits));
printf("All right 1: %s\n", toString(&bitsA));
printf("All left 1 : %s\n", toString(&bitsB));
printf("\n");
printf(" : ********\n");
printf("Starting : %s\n", toString(&bits));
printf("6,8,4 : %s\n", toString(&bitsC));
printf("6,8,-2 : %s\n", toString(&bitsD));
printf("6,8,31 : %s\n", toString(&bitsE));
printf("6,8,-31 : %s\n", toString(&bitsF));
gets(input);
}
ARRAYBITS rotateBits(ARRAYBITS *pOriginalBits, int startBit, int length, int shiftRightCount)
{
// 0-8 == 1, 9-16 == 2, 17-24 == 3
ARRAYBITS newBits;
int i = 0, j = 0;
int bytes = 0;
while (shiftRightCount < 0)
shiftRightCount += length;
shiftRightCount = shiftRightCount % length;
newBits.numBits = pOriginalBits->numBits;
if (pOriginalBits->numBits <= 0)
return newBits;
bytes = ((pOriginalBits->numBits -1) / 8) + 1;
newBits.buf = (unsigned char *)malloc(bytes);
memset(newBits.buf, 0, bytes);
for (i = 0; i < pOriginalBits->numBits; i++) {
j = (i < startBit || i >= startBit + length) ? i : ((i - startBit + shiftRightCount) % length) + startBit;
if (checkBit(pOriginalBits->buf, i))
{
setBit(newBits.buf, j, true);
}
}
return newBits;
}
void setBit(unsigned char *buf, int bit, bool isSet)
{
int charIndex = bit / 8;
unsigned char c = 1 << (bit & 0x07);
if (isSet)
buf[charIndex] |= c;
else
buf[charIndex] &= (c ^ 255);
}
bool checkBit(unsigned char *buf, int bit)
{
// address of char is (bit / 8), bit within char is (bit & 7)
int index = bit / 8;
int b = bit & 7;
int value = 1 << b;
return ((buf[index] & value) > 0);
}
ARRAYBITS fromString(char *onesAndZeros)
{
int i;
ARRAYBITS bits;
int charCount;
bits.numBits = strlen(onesAndZeros);
charCount = ((bits.numBits -1) / 8) + 1;
bits.buf = (unsigned char *)malloc(charCount);
memset(bits.buf, 0, charCount);
for (i = 0; i < bits.numBits; i++)
{
if (onesAndZeros[i] != '0')
setBit(bits.buf, i, true);
}
return bits;
}
char *toString(ARRAYBITS *pBits)
{
char *buf = (char *)malloc(pBits->numBits + 1);
int i;
for (i = 0; i < pBits->numBits; i++)
{
buf[i] = checkBit(pBits->buf, i) ? '1' : '0';
}
buf[i] = 0;
return buf;
}
I suggest you use bit-level operations (>>,<<,~,&,|) rather than wasting space using int. Even so, using an int array, to rotate, pass the left & right index of substring:
void rotate ( struct arrayBits a, int left , int right )
{
int i;
int first_bit;
if(*( a.buf + right ) == 1) first_bit = 1;
else first_bit = 0;
for( i = left+1 ; i <= right ; i++ )
{
*( a.buf + i )=*( a.buf + i - 1 );
}
*a.buf = first_bit;
}
Example:
If struct_array is 010101,
rotate (struct_array,0,5); => rotates whole string 1 int to right
o/p: 101010
rotate (struct_array,2,4); => rotates substring 1 int to right
o/p: 01 001 1
To reverse the bit array call the rotate() function on the substring, size_of_substring times.