Save and restart random chain (drand48) from checkpoint in C - c

I'm trying to write a program that gives the same result either if is executed entirely or if is stopped and restarted from some checkpoint. To do that I need to be able to repeat exactly the same random number sequence in any scenario. So, here a piece of code where I tried to do that, but of course, I'm not successful. Could you help me to fix this code?
int main(){
int i;
long int seed;
// Initial seed
srand48(3);
// Print 5 random numbers
for(i=0;i<5;i++) printf("%d %f\n",i,drand48());
// CHECKPOINT: HOW TO PROPERLY SET seed?
seed=mrand48(); // <--- FIXME
// 5 numbers more
for(i=5;i<10;i++) printf("%d %f\n",i,drand48());
// Restart from the CHECKPOINT.
srand48(seed);
// Last 5 numbers again
for(i=5;i<10;i++) printf("%d %f\n",i,drand48());
}

If you need to be able to resume the random number sequence, you can't let the drand48() package hide the seed values from you, so you need to use different functions from the package. Specifically, you should be calling:
double erand48(unsigned short xsubi[3]);
instead of:
double drand48(void);
and you'll keep an array of 3 unsigned short values around, and at each checkpoint, you'll record their values as part of the state. If you need to resume where things left off, you'll restore the values from the saved state into your array, and then go on your merry way.
This is also how you write library code that neither interferes with other code using the random number generators nor is interfered with by other code using the random number generators.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int main(void)
{
unsigned short seed[3] = { 0, 0, 3 };
// Print 5 random numbers
for (int i = 0; i < 5; i++)
printf("%d %f\n", i, erand48(seed));
// CHECKPOINT
unsigned short saved[3];
memmove(saved, seed, sizeof(seed));
// 5 numbers more
for (int i = 5; i < 10; i++)
printf("%d %f\n", i, erand48(seed));
// Restart from the CHECKPOINT.
memmove(seed, saved, sizeof(seed));
// Last 5 numbers again
for (int i = 5; i < 10; i++)
printf("%d %f\n", i, erand48(seed));
return 0;
}
Example run:
0 0.700302
1 0.122979
2 0.346792
3 0.290702
4 0.617395
5 0.059760
6 0.783933
7 0.352009
8 0.734377
9 0.124767
5 0.059760
6 0.783933
7 0.352009
8 0.734377
9 0.124767
Clearly, how you set the seed array initially is entirely up to you. You can easily allow the user to specify the seed value, and report the seed you're using so that they can do so. You might use some elements from the PID or the time of day and the sub-seconds component as a default seed, for example. Or you could access a random number device such as /dev/urandom and obtain 6 bytes of random value from that to use as the seed.
How can I allow the user to specify the seed value using only a long int? In this approach, it seems that the user need to define 3 numbers but I would like to ask only 1 number (like a safe prime) in the input file.
You can take a single number and split it up in any way you choose. I have a program that takes option -s to print the random seed, -S to set the seed from a long, and that sometimes splits the long into 3 unsigned short values when using a random Gaussian distribution generator. I mostly work on 64-bit systems, so I simply split the long into three 16-bit components; the code also compiles safely under 32-bit systems but leaves the third number in the seed as 0. Like this:
case 'q':
qflag = true;
break;
case 'r':
check_range(optarg, &min, &max);
perturber = ptb_uniform;
break;
case 's':
sflag = true;
break;
case 't':
delim = optarg;
break;
case 'S':
seed = strtol(optarg, 0, 0);
break;
case 'V':
err_version("PERTURB", &"#(#)$Revision: 1.6 $ ($Date: 2015/08/06 05:05:21 $)"[4]);
/*NOTREACHED*/
default:
err_usage(usestr);
/*NOTREACHED*/
}
}
if (sflag)
printf("Seed: %ld\n", seed);
if (gflag)
{
unsigned short g_seed[3] = { 0, 0, 0 };
g_seed[0] = (unsigned short)(seed & 0xFFFF);
g_seed[2] = (unsigned short)((seed >> 16) & 0xFFFF);
if (sizeof(seed) > 4)
{
/* Avoid 32-bit right shift on 32-bit platform */
g_seed[1] = (unsigned short)(((seed >> 31) >> 1) & 0xFFFF);
}
gaussian_init(&g_control, g_seed);
}
else
srand48(seed);
filter_anon(argc, argv, optind, perturb);
return 0;
}
For my purposes, it is OK (not ideal, but OK) to have the even more restricted seeding values for 32-bit. Yes, I could use unsigned long long and strtoull() etc instead, to get 64-bit numbers even on a 32-bit platform (though I'd have to convert that to a long to satisfy srand48() anyway. An alternative that I considered is to accept an argument -S xxxx:yyyy:zzzz with the three seed components set separately. I'd then have to modify the seed printing code as well as the parsing code. I use a separate program randseed to read numbers from /dev/urandom and format the result so it can be passed to programs which need a random seed:
$ randseed -b 8
0xF45820D2895B88CE
$

Related

Find two worst values and delete in sum

A microcontroller has the job to sample ADC Values (Analog to Digital Conversion). Since these parts are affected by tolerance and noise, the accuracy can be significantly increased by deleting the 4 worst values. The find and delete does take time, which is not ideal, since it will increase the cycle time.
Imagine a frequency of 100MHz, so each command of software does take 10ns to process, the more commands, the longer the controller is blocked from doing the next set of samples
So my goal is to do the sorting process as fast as possible for this i currently use this code, but this does only delete the two worst!
uint16_t getValue(void){
adcval[8] = {};
uint16_t min = 16383 //14bit full
uint16_t max = 1; //zero is physically almost impossible!
uint32_t sum = 0; //variable for the summing
for(uint8_t i=0; i<8;i++){
if(adc[i] > max) max = adc[i];
if(adc[i] < min) min = adc[i];
sum=sum+adcval[i];
}
uint16_t result = (sum-max-min)/6; //remove two worst and divide by 6
return result;
}
Now I would like to extend this function to delete the 4 worst values out of the 8 samples to get more precision. Any advice on how to do this?
Additionally, it would be wonderful to build an efficient function that finds the most deviating values, instead of the highest and lowest. For example, imagine the this two arrays
uint16_t adc1[8] {5,6,10,11,11,12,20,22};
uint16_t adc2[8] {5,6,7,7,10,11,15,16};
First case would gain precision by the described mechanism (delete the 4 worst). But the second case would have deleted the values 5 and 6 as well as 15 and 16. But this would theoretically make the calculation worse, since deleting 10,11,15,16 would be better. Is there any fast solution of deleting the 4 most deviating?
If your ADC is returning values from 5 to 16 14 bits and the voltage reference 3.3V, the voltage varies from 1mV to 3mV. It is very likely that it is the correct reading. It is very difficult to design good input circuit for 14 bits ADC.
It is better to run the running average. What is the running average? It is software low pass filter.
Blue are readings from the ADC, red -running average
Second signal is the very low amplitude sine wave (9-27mV - assuming 14 bits and 3.3Vref)
The algorithm:
static int average;
int running_average(int val, int level)
{
average -= average / level;
average += val * level;
return average / level;
}
void init_average(int val, int level)
{
average = val * level;
}
if the level is the power of 2. This version needs only 6 instructions (no branches) to calculate the average.
static int average;
int running_average(int val, int level)
{
average -= average >> level;
average += val << level;
return average >> level;
}
void init_average(int val, int level)
{
average = val << level;
}
I assume that average will no overflow. If yes you need to chose larger type
This answer is kinda of topic as it recommends a hardware solution but if performance is required and the MCU can't implement P__J__'s solution than this is your next best thing.
It seems you want to remove noise from your input signal. This can be done in software using DSP (digital signal processing) but it can also be done by configuring your hardware differently.
By adding the proper filter at the proper space before your ADC, it will be possible to remove much (outside) noise from your ADC output. (you can't of course go below a certain amount that is innate in the ADC but alas.)
There are several q&a on electronics.stackexchange.com.
One solution is adding a capacitor to filter some high frequency noise. As noted by DerStorm8
The Photon has another great solution here by suggesting RC, Sallen-Key and a cascade of Sallen-Key filters for a continuous signal filter.
Here (ADN007) is a Analog Design Note from Microchip on "Techniques that Reduce System Noise in ADC Circuits"
It may seem that designing a low noise, 12-bit Analog-to-Digital
Converter (ADC) board or even a 10-bit board is easy. This is
true, unless one ignores the basics of low noise design. For
instance, one would think that most amplifiers and resistors work
effectively in 12-bit or 10-bit environments. However, poor device
selection becomes a major factor in the success or failure of the
circuit. Another, often ignored, area that contributes a great deal
of noise, is conducted noise. Conducted noise is already in the
circuit board by the time the signal arrives at the input of the
ADC. The most effective way to remove this noise is by using a
low-pass (anti-aliasing) filter prior to the ADC. Including by-pass
capacitors and using a ground plane will also eliminate this type
of noise. A third source of noise is radiated noise. The major
sources of this type of noise are Electromagnetic Interference
(EMI) or capacitive coupling of signals from trace-to-trace.
If all three of these issues are addressed, then it is true that
designing a low noise 12-bit ADC board is easy.
And their recommended solution path:
It is easy to design a true 12-bit ADC system by using a few
key low noise guidelines. First, examine your devices (resistors
and amplifiers) to make sure they are low noise. Second, use a
ground plane whenever possible. Third, include a low-pass filter
in the signal path if you are changing the signal from analog to
digital. Finally, and always, include by-pass capacitors. These
capacitors not only remove noise but also foster circuit stability.
Here is a good paper by Analog Devices on input noise. They note in here that "there are some instances where input noise can actually be helpful in achieving higher resolution."
All analog-to-digital converters (ADCs) have a certain amount of input-referred noise—modeled as a noise source connected in series with the input of a noise-free ADC. Input-referred noise is not to be confused with quantization noise, which is only of interest when an ADC is processing time-varying signals. In most cases, less input noise is better; however, there are some instances where input noise can actually be helpful in achieving higher resolution. If this doesn’t seem to make sense right now, read on to find out how some noise can be good noise.
Given that you have a fixed size array, a hard-coded sorting network should be able to correctly sort the entire array with only 19 comparisons. Currently you have 8+2*8=24 comparisons already, although it is possible that the compiler unrolls the loop, leaving you with 16 comparisons. It is conceivable that, depending on the microcontroller hardware, a sorting network can be implemented with some degree of parallelism -- perhaps you also have to query the adc values sequentially which would give you opportunity to pre-sort them, while waiting for the comparison.
An optimal sorting network should be searchable online. Wikipedia has some pointers.
So, you would end up with some code like this:
sort_data(adcval);
return (adcval[2]+adcval[3]+adcval[4]+adcval[5])/4;
Update:
As you can take from this picture (source) of optimal sorting networks, a complete sort takes 19 comparisons. However 3 of those are not strictly needed if you only want to extract the middle 4 values. So you get down to 16 comparisons.
to delete the 4 worst values out of the 8 samples
The methods are described on geeksforgeeks k largest(or smallest) elements in an array and you can implement the best method that suits you.
I decided to use this good site to generate best sorting algorithm with SWAP() macros needed to sort the array of 8 elements. Then I created a small C program that will test any combination of 8 element array on my sorting function. Then, because we only care of groups of 4 elements, I did something bruteforce - for each of the SWAP() macros I tried to comment the macro and see if the program still succeeds. I could comment 5 SWAP macros, leaving 14 comparisons needed to identify the smallest 4 elements in the array of 8 samples.
/**
* Sorts the array, but only so that groups of 4 matter.
* So group of 4 smallest elements and 4 biggest elements
* will be sorted ok.
* s[0]...s[3] will have lowest 4 elements
* so they have to be "deleted"
* s[4]...s[7] will have the highest 4 values
*/
void sort_but_4_matter(int s[8]) {
#define SWAP(x, y) do { \
if (s[x] > s[y]) { \
const int t = s[x]; \
s[x] = s[y]; \
s[y] = t; \
} \
} while(0)
SWAP(0, 1);
//SWAP(2, 3);
SWAP(0, 2);
//SWAP(1, 3);
//SWAP(1, 2);
SWAP(4, 5);
SWAP(6, 7);
SWAP(4, 6);
SWAP(5, 7);
//SWAP(5, 6);
SWAP(0, 4);
SWAP(1, 5);
SWAP(1, 4);
SWAP(2, 6);
SWAP(3, 7);
//SWAP(3, 6);
SWAP(2, 4);
SWAP(3, 5);
SWAP(3, 4);
#undef SWAP
}
/* -------- testing code */
#include <assert.h>
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
int cmp_int(const void *a, const void *b) {
return *(const int*)a - *(const int*)b;
}
void printit_arr(const int *arr, size_t n) {
printf("{");
for (size_t i = 0; i < n; ++i) {
printf("%d", arr[i]);
if (i != n - 1) {
printf(" ");
}
}
printf("}");
}
void printit(const char *pre, const int arr[8],
const int in[8], const int res[4]) {
printf("%s: ", pre);
printit_arr(arr, 8);
printf(" ");
printit_arr(in, 8);
printf(" ");
printit_arr(res, 4);
printf("\n");
}
int err = 0;
void test(const int arr[8], const int res[4]) {
int in[8];
memcpy(in, arr, sizeof(int) * 8);
sort_but_4_matter(in);
// sort for memcmp below
qsort(in, 4, sizeof(int), cmp_int);
if (memcmp(in, res, sizeof(int) * 4) != 0) {
printit("T", arr, in, res);
err = 1;
}
}
void test_all_combinations() {
const int result[4] = { 0, 1, 2, 3 }; // sorted
const size_t n = 8;
int num[8] = { 0, 1, 2, 3, 4, 5, 6, 7 };
for (size_t j = 0; j < n; j++) {
for (size_t i = 0; i < n-1; i++) {
int temp = num[i];
num[i] = num[i+1];
num[i+1] = temp;
test(num, result);
}
}
}
int main() {
test_all_combinations();
return err;
}
Tested on godbolt. The sort_but_4_matter with gcc -O2 on x86_64 compiles to less then 100 instruction.

Generating random values without time.h

I want to generate random numbers repeatedly without using the time.h library. I saw another post regarding use the
srand(getpid());
however that doesn't seem to work for me getpid hasn't been declared. Is this because I'm missing the library for it? If it is I need to work out how to randomly generate numbers without using any other libraries than the ones I currently have.
#include <stdio.h>
#include <stdlib.h>
int main(void) {
int minute, hour, day, month, year;
srand(getpid());
minute = rand() % (59 + 1 - 0) + 0;
hour = rand() % (23 + 1 - 0) + 0;
day = rand() % (31 + 1 - 1) + 1;
month = rand() % (12 + 1 - 1) + 1;
year = 2018;
printf("Transferred successfully at %02d:%02d on %02d/%02d/%d\n", hour,
minute, day, month, year);
return 0;
}
NB: I can only use libraries <stdio.h> and <stdlib.h> and <string.h> — strict guidelines for an assignment.
getpid hasn't been declared.
No, because you haven't included the <unistd.h> header where it is declared (and according to your comment, you cannot use it, because you're restricted to using <stdlib.h>, <string.h>, and <stdio.h>).
In that case, I would use something like
#include <stdlib.h>
#include <stdio.h>
static int randomize_helper(FILE *in)
{
unsigned int seed;
if (!in)
return -1;
if (fread(&seed, sizeof seed, 1, in) == 1) {
fclose(in);
srand(seed);
return 0;
}
fclose(in);
return -1;
}
static int randomize(void)
{
if (!randomize_helper(fopen("/dev/urandom", "r")))
return 0;
if (!randomize_helper(fopen("/dev/arandom", "r")))
return 0;
if (!randomize_helper(fopen("/dev/random", "r")))
return 0;
/* Other randomness sources (binary format)? */
/* No randomness sources found. */
return -1;
}
and a simple main() to output some pseudorandom numbers:
int main(void)
{
int i;
if (randomize())
fprintf(stderr, "Warning: Could not find any sources for randomness.\n");
for (i = 0; i < 10; i++)
printf("%d\n", rand());
return EXIT_SUCCESS;
}
The /dev/urandom and /dev/random character devices are available in Linux, FreeBSD, macOS, iOS, Solaris, NetBSD, Tru64 Unix 5.1B, AIX 5.2, HP-UX 11i v2, and /dev/random and /dev/arandom on OpenBSD 5.1 and later.
As usual, it looks like Windows does not provide any such randomness sources: Windows C programs must use proprietary Microsoft interfaces instead.
The randomize_helper() returns nonzero if the input stream is NULL, or if it cannot read an unsigned int from it. If it can read an unsigned int from it, it is used to seed the standard pseudorandom number generator you can access using rand() (which returns an int between 0 and RAND_MAX, inclusive). In all cases, randomize_helper() closes non-NULL streams.
You can add other binary randomness sources to randomize() trivially.
If randomize() returns 0, rand() should return pseudorandom numbers. Otherwise, rand() will return the same default sequence of pseudorandom numbers. (They will still be "random", but the same sequence will occur every time you run the program. If randomize() returns 0, the sequence will be different every time you run the program.)
Most standard C rand() implementations are linear congruental pseudorandom number generators, often with poor choices of parameters, and as a result, are slowish, and not very "random".
For non-cryptographic work, I like to implement one of the Xorshift family of functions, originally by George Marsaglia. They are very, very fast, and reasonably random; they pass most of the statistical randomness tests like the diehard tests.
In OP's case, the xorwow generator could be used. According to current C standards, unsigned int is at least 32 bits, so we can use that as the generator type. Let's see what implementing one to replace the standard srand()/rand() would look like:
#include <stdlib.h>
#include <stdio.h>
/* The Xorwow PRNG state. This must not be initialized to all zeros. */
static unsigned int prng_state[5] = { 1, 2, 3, 4, 5 };
/* The Xorwow is a 32-bit linear-feedback shift generator. */
#define PRNG_MAX 4294967295u
unsigned int prng(void)
{
unsigned int s, t;
t = prng_state[3] & PRNG_MAX;
t ^= t >> 2;
t ^= t << 1;
prng_state[3] = prng_state[2];
prng_state[2] = prng_state[1];
prng_state[1] = prng_state[0];
s = prng_state[0] & PRNG_MAX;
t ^= s;
t ^= (s << 4) & PRNG_MAX;
prng_state[0] = t;
prng_state[4] = (prng_state[4] + 362437) & PRNG_MAX;
return (t + prng_state[4]) & PRNG_MAX;
}
static int prng_randomize_from(FILE *in)
{
size_t have = 0, n;
unsigned int seed[5] = { 0, 0, 0, 0, 0 };
if (!in)
return -1;
while (have < 5) {
n = fread(seed + have, sizeof seed[0], 5 - have, in);
if (n > 0 && ((seed[0] | seed[1] | seed[2] | seed[3] | seed[4]) & PRNG_MAX) != 0) {
have += n;
} else {
fclose(in);
return -1;
}
}
fclose(in);
prng_seed[0] = seed[0] & PRNG_MAX;
prng_seed[1] = seed[1] & PRNG_MAX;
prng_seed[2] = seed[2] & PRNG_MAX;
prng_seed[3] = seed[3] & PRNG_MAX;
prng_seed[4] = seed[4] & PRNG_MAX;
/* Note: We might wish to "churn" the pseudorandom
number generator state, to call prng()
a few hundred or thousand times. For example:
for (n = 0; n < 1000; n++) prng();
This way, even if the seed has clear structure,
for example only some low bits set, we start
with a PRNG state with set and clear bits well
distributed.
*/
return 0;
}
int prng_randomize(void)
{
if (!prng_randomize_from(fopen("/dev/urandom", "r")))
return 0;
if (!prng_randomize_from(fopen("/dev/arandom", "r")))
return 0;
if (!prng_randomize_from(fopen("/dev/random", "r")))
return 0;
/* Other sources? */
/* No randomness sources found. */
return -1;
}
The corresponding main() to above would be
int main(void)
{
int i;
if (prng_randomize())
fprintf(stderr, "Warning: No randomness sources found!\n");
for (i = 0; i < 10; i++)
printf("%u\n", prng());
return EXIT_SUCCESS;
}
Note that PRNG_MAX has a dual purpose. On one hand, it tells the maximum value prng() can return -- which is an unsigned int, not int like rand(). On the other hand, because it must be 232-1 = 4294967295, we also use it to ensure the temporary results when generating the next pseudorandom number in the sequence remain 32-bit. If the uint32_t type, declared in stdint.h or inttypes.h were available, we could use that and drop the masks (& PRNG_MAX).
Note that the prng_randomize_from() function is written so that it still works, even if the randomness source cannot provide all requested bytes at once, and returns a "short count". Whether this occurs in practice is up to debate, but I prefer to be certain. Also note that it does not accept the state if it is all zeros, as that is the one single prohibited initial seed state for the Xorwow PRNG.
You can obviously use both srand()/rand() and prng()/prng_randomize() in the same program. I wrote them so that the Xorwow generator functions all start with prng.
Usually, I do put the PRNG implementation into a header file, so that I can easily test it (to verify it works) by writing a tiny test program; but also so that I can switch the PRNG implementation simply by switching to another header file. (In some cases, I put the PRNG state into a structure, and have the caller provide a pointer to the state, so that any number of PRNGs can be used concurrently, independently of each other.)
however that doesn't seem to work for me getpid hasn't been declared.
That's because you need to include the headers for getpid():
#include <sys/types.h>
#include <unistd.h>
Another option is to use time() to seed (instead of getpid()):
srand((unsigned int)time(NULL));
As other answer pointed, you need to include the unistd.h header. If you don't want to do that then put the declaration of getpid() above main(). Read the manual page of getpid() here http://man7.org/linux/man-pages/man2/getpid.2.html
One approach may be
#include <stdio.h>
#include <stdlib.h>
pid_t getpid(void); /* put the declrataion of getpid(), if don't want to include the header */
int main(void) {
/* .. some code .. */
return 0;
}
Or you can use time() like
srand((unsigned int)time(NULL));

Why is recursion taking so long?

In using recursion to calculate the nth number of the fibonacci sequence, I have written this simple program:
#include <stdio.h>
#include <stdlib.h>
#include <limits.h>
unsigned int long long fibonacci(unsigned int number);
//game of craps
int main(int argc, char** argv)
{
for(int n = 1; n <= 100; n++)
{
printf("%llu\n", fibonacci(n));
}
return (EXIT_SUCCESS);
}
unsigned long long int fibonacci(unsigned int number)
{
if (number == 0 || number == 1)
{
return number;
}
else
{
return fibonacci(number - 2) + fibonacci(number - 1);
}
}
where each call to the n+1 number in the sequence doubles the number of function calls the program has to run. Therefore the number of calls being made to the recursive function is something of 2^n, or exponential complexity. Understood. But where is all of the computing power going? once the nth number in the sequence starts to hit 40, the computer starts taking noticeable time to compute the result where at n = 47 its take 30+ seconds. However my computer shows that I'm only using 21 percent of cpu power. I'm using NetBeans IDE to run the program. It's a quad core system.
the number of calls being made to the recursive function is something of 2^n, or exponential complexity. Understood.
I'm not sure you do entirely understand this, since you seem surprised about how slow it becomes around n=40, and n=47.
With a complexity of 2^n, and an n of 40, that would be 240, or 1,099,511,627,776, or about 1 trillion operations. If your computer can run about one of these operations per nanosecond, i.e. 1 billion operations per second, it would take 1000 seconds to finish.
Consider if n was only 30. 230 is 1,073,741,824, which would take only about 1 second to do on that same computer.
As has been mentioned, you're only using one core. You could parallelize, but that won't help much. Use four cores instead of one, and my n=40 example will still take 250 seconds. Go up to n=42 and you're back to 1000 seconds, because parallelizing at best multiplies your performance, but an algorithm like this grows exponentially.
the posted code contains some extreme over complexity.
even a long long unsigned int cannot contain a Fibonacci value
number 100 (or even close to it)
Suggest using a very simple program to start, one that calculates the Fibonacci sequence. Then use that program to determine how to display the results.
The following program calculates the numbers, is very fast, but still has the problem of overflow of a long long unsigned int
#include <stdio.h> // printf()
int main( void )
{
long long unsigned currentNum = 1;
long long unsigned priorNum = 1;
printf( "1\n1\n" );
for (size_t i = 2; i < 100; i++ )
{
long long unsigned newNum = currentNum+priorNum;
printf( "%llu\n", newNum );
priorNum = currentNum;
currentNum = newNum;
}
}
On my linux 86-64 computer, here are the last few lines of the output, showing the overflow problem.
99194853094755497
160500643816367088
259695496911122585
420196140727489673
679891637638612258
1100087778366101931
1779979416004714189
2880067194370816120
4660046610375530309
7540113804746346429
12200160415121876738
1293530146158671551
13493690561280548289
14787220707439219840
9834167195010216513
6174643828739884737
16008811023750101250
3736710778780434371
So, why is recursion taking so long?
because of the huge number of recursions and the handling of the overflows
The above suggested code eliminates the recursions, but not the overflows and it takes less than a second (on my computer) to run.
You won't exploit a quad core system if you have a single-threaded program.
It will run on one core only, so the 21/25% CPU usage is realistic.
A way to use it all would be, first of all not using recursion as it makes it annoying to do, and when you have a for/while loop split it into 4 while loops and put each of them in a new thread. Then you'll have to manage synchronization in order to print the message properly, but it's not even that hard. You could store all the results in an array and then print it when all the threads are done.
Building on the answer made by #user3629249, you can get rid of the overflows he mentioned by using the infinite precision arithmetic library provided by GMP.
e.g.
#include <stdio.h> // printf
#include <stdlib.h> // free
#include <gmp.h> // mpz_t
int main( void )
{
mpz_t prevNum, currNum, tempNum, counter;
mpz_init_set_si(prevNum, 0);
mpz_init_set_si(currNum, 1);
mpz_init_set_si(tempNum, 1);
mpz_init_set_si(counter, 1);
printf( "0: 0\n" );
while (1) {
char *tempNumRepr = mpz_get_str(NULL, 10, tempNum);
char *counterRepr = mpz_get_str(NULL, 10, counter);
printf("%s: %s\n", counterRepr, tempNumRepr);
free(tempNumRepr);
free(counterRepr);
mpz_add(tempNum, currNum, prevNum); // tempNum = currNum + prevNum;
mpz_add_ui(counter, counter, 1); // counter = counter + 1;
mpz_set(prevNum, currNum); // prevNum = currNum;
mpz_set(currNum, tempNum); // currNum = tempNum;
}
mpz_clear(prevNum);
mpz_clear(currNum);
mpz_clear(tempNum);
mpz_clear(counter);
return EXIT_SUCCESS;
};
To compile it, ensure that you have libgmp installed, type:
~$ gcc fib.c -lgmp
You get massive fibonacci values pretty fast:
~$ ./a.out
0: 0
1: 1
2: 1
3: 2
4: 3
5: 5
6: 8
7: 13
8: 21
9: 34
...
90: 2880067194370816120
91: 4660046610375530309
92: 7540113804746346429
93: 12200160415121876738
94: 19740274219868223167
95: 31940434634990099905
96: 51680708854858323072
97: 83621143489848422977
98: 135301852344706746049
99: 218922995834555169026
100: 354224848179261915075
...
142: 212207101440105399533740733471
143: 343358302784187294870275058337
144: 555565404224292694404015791808
145: 898923707008479989274290850145
146: 1454489111232772683678306641953
147: 2353412818241252672952597492098
148: 3807901929474025356630904134051
149: 6161314747715278029583501626149
150: 9969216677189303386214405760200
...
10456: 6687771891046976665010914682715972428661740561209776353485935351631179302708216108795962659308263419533746676628535531789045787219342206829688433844719175383255599341828410480942962469553971997586487609675800755252584139702413749597015823849849046700521430415467867019518212926720410106893075072562394664597041033593563521410003073230903292197734713471051090595503533547412747118747787351929732433449493727418908972479566909080954709569018619548197645271462668017096925677064951824250666293199593131718849011440475925874263429880250725807157443918222920142864819346465587051597207982477956741428300547495546275347374411309127960079792636429623948756731669388275421014167909883947268371246535572766045766175917299574719971717954980856956555916099403979976768699108922030154574061373884317374443228652666763423361895311742060974910298682465051864682016439317005971937944787596597197162234588349001773183227535867183191706435572614767923270023480287832648770215573899455920695896713514952891911913499762717737021116746179317675622780792638129991728650763618970292905899648572351513919065201266611540504973510404007895858009291738402611754822294670524761118059571137973416151185102238975390542996959456114838498320921216851752236455715812273599551395186676228882752252829522673168259864505917922994675966393982705428427387550834530918600733123354437191268657802903434440996622861582962869292133202292740984119730918997492224957849300327645752441866958526558379656521799598935096546592129670888574358354955519855060127168291877171959996776081517513455753528959306416265886428706197994064431298142841481516239689015446304286858347321708226391039390175388745315544138793021359869227432464706950061238138314080606377506673283324908921190615421862717588664540813607678946107283312579595718137450873566434040358736923152893920579043838335105796035360841757227288861017982575677839192583578548045589322945
...
Use CTRL+C to stop the program.

Fisher Yates algorithm gives back same order of numbers in parallel started programs when seeded over the system time

I start several C / C++ programs in parallel, which rely on random numbers. Fairly new to this topic, I heard that the seed should be done over the time.
Furthermore, I use the Fisher Yates Algorithm to get a list with unique random shuffled values. However, starting the program twice in parallel gives back the same results for both lists.
How can I fix this? Can I use a different, but still relient seed?
My simple test code for this looks like this:
#include <stdint.h>
#include <stdlib.h>
#include <stdio.h>
#include <math.h>
#include <time.h>
static int rand_int(int n) {
int limit = RAND_MAX - RAND_MAX % n;
int rnd;
do {
rnd = rand();
}
while (rnd >= limit);
return rnd % n;
}
void shuffle(int *array, int n) {
int i, j, tmp;
for (i = n - 1; i > 0; i--) {
j = rand_int(i + 1);
tmp = array[j];
array[j] = array[i];
array[i] = tmp;
}
}
int main(int argc,char* argv[]){
srand(time(NULL));
int x = 100;
int randvals[100];
for(int i =0; i < x;i++)
randvals[i] = i;
shuffle(randvals,x);
for(int i=0;i < x;i++)
printf("%d %d \n",i,randvals[i]);
}
I used the implementation for the fisher yates algorithm from here:
http://www.sanfoundry.com/c-program-implement-fisher-yates-algorithm-array-shuffling/
I started the programs in parallel like this:
./randomprogram >> a.txt & ./randomprogram >> b.txt
and then compared both text files, which had the same content.
The end application is for data augmentation in the deep learning field. The machine runs Ubuntu 16.04 with C++11.
You're getting the same results due to how you're seeding the RNG:
srand(time(NULL));
The time function returns the time in seconds since the epoch. If two instances of the program start during the same second (which is likely if start them in quick succession) then both will use the same seed and get the same set of random values.
You need to add more entropy to your seed. A simple way of doing this is to bitwise-XOR the process ID with the time:
srand(time(NULL) ^ getpid());
As I mentioned in a comment, I like to use a Xorshift* pseudo-random number generator, seeded from /dev/urandom if present, otherwise using POSIX.1 clock_gettime() and getpid() to seed the generator.
It is good enough for most statistical work, but obviously not for any kind of security or cryptographic purposes.
Consider the following xorshift64.h inline implementation:
#ifndef XORSHIFT64_H
#define XORSHIFT64_H
#include <stdlib.h>
#include <unistd.h>
#include <stdint.h>
#include <time.h>
#ifndef SEED_SOURCE
#define SEED_SOURCE "/dev/urandom"
#endif
typedef struct {
uint64_t state[1];
} prng_state;
/* Mixes state by generating 'rounds' pseudorandom numbers,
but does not store them anywhere. This is often done
to ensure a well-mixed state after seeding the generator.
*/
static inline void prng_skip(prng_state *prng, size_t rounds)
{
uint64_t state = prng->state[0];
while (rounds-->0) {
state ^= state >> 12;
state ^= state << 25;
state ^= state >> 27;
}
prng->state[0] = state;
}
/* Returns an uniform pseudorandom number between 0 and 2**64-1, inclusive.
*/
static inline uint64_t prng_u64(prng_state *prng)
{
uint64_t state = prng->state[0];
state ^= state >> 12;
state ^= state << 25;
state ^= state >> 27;
prng->state[0] = state;
return state * UINT64_C(2685821657736338717);
}
/* Returns an uniform pseudorandom number [0, 1), excluding 1.
This carefully avoids the (2**64-1)/2**64 bias on 0,
but assumes that the double type has at most 63 bits of
precision in the mantissa.
*/
static inline double prng_one(prng_state *prng)
{
uint64_t u;
double d;
do {
do {
u = prng_u64(prng);
} while (!u);
d = (double)(u - 1u) / 18446744073709551616.0;
} while (d == 1.0);
return d;
}
/* Returns an uniform pseudorandom number (-1, 1), excluding -1 and +1.
This carefully avoids the (2**64-1)/2**64 bias on 0,
but assumes that the double type has at most 63 bits of
precision in the mantissa.
*/
static inline double prng_delta(prng_state *prng)
{
uint64_t u;
double d;
do {
do {
u = prng_u64(prng);
} while (!u);
d = ((double)(u - 1u) - 9223372036854775808.0) / 9223372036854775808.0;
} while (d == -1.0 || d == 1.0);
return d;
}
/* Returns an uniform pseudorandom integer between min and max, inclusive.
Uses the exclusion method to ensure uniform distribution.
*/
static inline uint64_t prng_range(prng_state *prng, const uint64_t min, const uint64_t max)
{
if (min != max) {
const uint64_t basis = (min < max) ? min : max;
const uint64_t range = (min < max) ? max-min : min-max;
uint64_t mask = range;
uint64_t u;
/* In range, all bits up to the higest bit set in range, must be set. */
mask |= mask >> 1;
mask |= mask >> 2;
mask |= mask >> 4;
mask |= mask >> 8;
mask |= mask >> 16;
mask |= mask >> 32;
/* In all cases, range <= mask < 2*range, so at worst case,
(mask = 2*range-1), this excludes at most 50% of generated values,
on average. */
do {
u = prng_u64(prng) & mask;
} while (u > range);
return u + basis;
} else
return min;
}
static inline void prng_seed(prng_state *prng)
{
#if _POSIX_TIMERS-0 > 0
struct timespec now;
#endif
FILE *src;
/* Try /dev/urandom. */
src = fopen(SEED_SOURCE, "r");
if (src) {
int tries = 16;
while (tries-->0) {
if (fread(prng->state, sizeof prng->state, 1, src) != 1)
break;
if (prng->state[0]) {
fclose(src);
return;
}
}
fclose(src);
}
#if _POSIX_TIMERS-0 > 0
#if _POSIX_MONOTONIC_CLOCK-0 > 0
if (clock_gettime(CLOCK_MONOTONIC, &now) == 0) {
prng->state[0] = (uint64_t)((uint64_t)now.tv_sec * UINT64_C(60834327289))
^ (uint64_t)((uint64_t)now.tv_nsec * UINT64_C(34958268769))
^ (uint64_t)((uint64_t)getpid() * UINT64_C(2772668794075091))
^ (uint64_t)((uint64_t)getppid() * UINT64_C(19455108437));
if (prng->state[0])
return;
} else
#endif
if (clock_gettime(CLOCK_REALTIME, &now) == 0) {
prng->state[0] = (uint64_t)((uint64_t)now.tv_sec * UINT64_C(60834327289))
^ (uint64_t)((uint64_t)now.tv_nsec * UINT64_C(34958268769))
^ (uint64_t)((uint64_t)getpid() * UINT64_C(2772668794075091))
^ (uint64_t)((uint64_t)getppid() * UINT64_C(19455108437));
if (prng->state[0])
return;
}
#endif
prng->state[0] = (uint64_t)((uint64_t)time(NULL) * UINT64_C(60834327289))
^ (uint64_t)((uint64_t)clock() * UINT64_C(34958268769))
^ (uint64_t)((uint64_t)getpid() * UINT64_C(2772668794075091))
^ (uint64_t)((uint64_t)getppid() * UINT64_C(19455108437));
if (!prng->state[0])
prng->state[0] = (uint64_t)UINT64_C(16233055073);
}
#endif /* XORSHIFT64_H */
If it can seed the state from SEED_SOURCE, it is used as-is. Otherwise, if POSIX.1 clock_gettime() is available, it is used (CLOCK_MONOTONIC, if possible; otherwise CLOCK_REALTIME). Otherwise, time (time(NULL)), CPU time spent thus far (clock()), process ID (getpid()), and parent process ID (getppid()) are used to seed the state.
If you wanted the above to also run on Windows, you'd need to add a few #ifndef _WIN32 guards, and either omit the process ID parts, or replace them with something else. (I don't use Windows myself, and cannot test such code, so I omitted such from above.)
The idea is that you can include the above file, and implement other pseudo-random number generators in the same format, and choose between them by simply including different files. (You can include multiple files, but you'll need to do some ugly #define prng_state prng_somename_state, #include "somename.h", #undef prng_state hacking to ensure unique names for each.)
Here is an example of how to use the above:
#include <stdlib.h>
#include <inttypes.h>
#include <stdint.h>
#include <stdio.h>
#include "xorshift64.h"
int main(void)
{
prng_state prng1, prng2;
prng_seed(&prng1);
prng_seed(&prng2);
printf("Seed 1 = 0x%016" PRIx64 "\n", prng1.state[0]);
printf("Seed 2 = 0x%016" PRIx64 "\n", prng2.state[0]);
printf("After skipping 16 rounds:\n");
prng_skip(&prng1, 16);
prng_skip(&prng2, 16);
printf("Seed 1 = 0x%016" PRIx64 "\n", prng1.state[0]);
printf("Seed 2 = 0x%016" PRIx64 "\n", prng2.state[0]);
return EXIT_SUCCESS;
}
Obviously, initializing two PRNGs like this is problematic in the fallback case, because it basically relies on clock() yielding different values for consecutive calls (so expects each call to take at least 1 millisecond of CPU time).
However, even a small change in the seeds thus generated is sufficient to yield very different sequences. I like to generate and discard (skip) a number of initial values to ensure the generator state is well mixed:
Seed 1 = 0x8a62585b6e71f915
Seed 2 = 0x8a6259a84464e15f
After skipping 16 rounds:
Seed 1 = 0x9895f664c83ad25e
Seed 2 = 0xa3fd7359dd150e83
The header also implements 0 <= prng_u64() < 2**64, 0 <= prng_one() < 1, -1 < prng_delta() < +1, and min <= prng_range(,min,max) <= max, which should be uniform.
I use the above Xorshift64* variant for tasks where a lot of quite uniform pseudorandom numbers are needed, so the functions also tend to use the faster methods (like max. 50% average exclusion rate rather than 64-bit modulus operation, and so on) (of those that I know of).
Additionally, if you require repeatability, you can simply save a randomly-seeded prng_state structure (a single uint64_t), and load it later, to reproduce the exact same sequence. Just remember to only do the skipping (generate-and-discard) only after randomly seeding, not after loading a new seed from a file.
Converting rather copious comments into an answer.
If two programs are started in the same second, they'll both have the same sequence of random numbers.
Consider whether you need to use a better random number generator than the rand()/srand() duo — that is usually only barely random (better than nothing, but not by a large margin). Do NOT use them for cryptography.
I asked about platform; you responded Ubuntu 16.04 LTS.
Use /dev/urandom or /dev/random to get some random bytes for the seed.
On many Unix-like platforms, there's a device /dev/random — on Linux, there's also a slightly lower-quality device /dev/urandom which won't block whereas /dev/random might. Systems such as macOS (BSD) have /dev/urandom as a synonym for /dev/random for Linux compatibility. You can open it and read 4 bytes (or the relevant number of bytes) of random data, and use that as a seed for the PRNG of your choice.
I often use the drand48() set of functions because they are in POSIX and were in System V Unix. They're usually adequate for my needs.
Look at the manuals across platforms; there are often other random number generators. C++11 provides high-quality PRNG — the header <random> has a number of different ones, such as the MT 19937 (Mersenne Twister). MacOS Sierra (BSD) has random(3) and arc4random(3) as alternatives to rand() – as well as drand48() et al.
Another possibility on Linux is simply to keep a connection to /dev/urandom open, reading more bytes when you need them. However, that gives up any chance of replaying a random sequence. The PRNG systems have the merit of allowing you to replay the same sequence again by recording and setting the random seed that you use. By default, grab a seed from /dev/urandom, but if the user requests it, take a seed from the command line, and report the seed used (at least on request).

How do I create a "twirly" in a C program task?

Hey guys I have created a program in C that tests all numbers between 1 and 10000 to check if they are perfect using a function that determines whether a number is perfect. Once it finds these it prints them to the user, they are 6, 28, 496 and 8128. After this the program then prints out all the factors of each perfect number to the user. This is all fine. Here is my problem.
The final part of my task asks me to:
"Use a "twirly" to indicate that your program is happily working away. A "twirly" is the following characters printed over the top of each other in the following order: '|' '/' '-' '\'. This has the effect of producing a spinning wheel - ie a "twirly". Hint: to do this you can use \r (instead of \n) in printf to give a carriage return only (instead of a carriage return linefeed). (Note: this may not work on some systems - you do not have to do it this way.)"
I have no idea what a twirly is or how to implement one. My tutor said it has something to do with the sleep and delay functions which I also don't know how to use. Can anyone help me with this last stage, it sucks that all my coding is complete but I can't get this "twirly" thing to work.
if you want to simultaneously perform the task of
Testing the numbers and
Display the twirly on screen
while the process goes on then you better look into using threads. using POSIX threads you can initiate the task on a thread and the other thread will display the twirly to the user on terminal.
#include<stdlib.h>
#include<pthread.h>
int Test();
void Display();
int main(){
// create threads each for both tasks test and Display
//call threads
//wait for Test thread to finish
//terminate display thread after Test thread completes
//exit code
}
Refer chapter 12 for threads
beginning linux programming ebook
Given the program upon which the user is "waiting", I believe the problem as stated and the solutions using sleep() or threads are misguided.
To produce all the perfect numbers below 10,000 using C on a modern personal computer takes about 1/10 of a second. So any device to show the computer is "happily working away" would either never be seen or would significanly intefere with the time it takes to get the job done.
But let's make a working twirly for perfect number search anyway. I've left off printing the factors to keep this simple. Since 10,000 is too low to see the twirly in action, I've upped the limit to 100,000:
#include <stdio.h>
#include <string.h>
int main()
{
const char *twirly = "|/-\\";
for (unsigned x = 1; x <= 100000; x++)
{
unsigned sum = 0;
for (unsigned i = 1; i <= x / 2; i++)
{
if (x % i == 0)
{
sum += i;
}
}
if (sum == x)
{
printf("%d\n", x);
}
printf("%c\r", twirly[x / 2500 % strlen(twirly)]);
}
return 0;
}
No need for sleep() or threads, just key it into the complexity of the problem itself and have it update at reasonable intervals.
Now here's the catch, although the above works, the user will never see a fifth perfect number pop out with a 100,000 limit and even with a 100,000,000 limit, which should produce one more, they'll likely give up as this is a bad (slow) algorithm for finding them. But they'll have a twirly to watch.
i as integer
loop i: 1 to 10000
loop j: 1 to i/2
sum as integer
set sum = 0
if i%j == 0
sum+=j
return sum==i
if i%100 == 0
str as character pointer
set *str = "|/-\\"
set length = 4
print str[p] using "%c\r" as format specifier
Increment p and assign its modulo by len to p

Resources