I'm developing some library in C that can be used by various user applications.
The library should be completely "transparent" - a user application can init it and finalize,
and it's not supposed to see any change in the running application.
The problem is - I'm using C srand()/rand() functions in the library initialization,
which means that the library does affect user's application - if a user generates random numbers, they will be affected by the fact that rand() was already called.
So, can anyone point to some simple non-GPL alternative to rand() random number generator in C?
It doesn't have to be really strong - I'n not doing any crypto with the numbers.
I was thinking to write some small and really simple generator (something like take time and XOR with something and do something with some prime number and bla bla bla), but I was wondering if someone has a pointer to a more decent generator.
It generates the next number by keeping some state and modifying the state every time you call the function. Such a function is called a pseudorandom number generator. An old method of creating a PRNG is the linear congruential generator, which is easy enough:
static int rand_state;
int rand(void)
{
rand_state = (rand_state * 1103515245 + 12345) & 0x7fffffff;
return rand_state;
}
As you can see, this method allows you to predict the next number in the series if you know the previous number. There are more sophisticated methods.
Various types of pseudorandom number generators have been designed for specific purposes. There are secure PRNGs which are slow but hard to predict even if you know how they work, and there are big PRNGs like Mersenne Twister which have nice distribution properties and are therefore useful for writing Monte Carlo simulations.
As a rule of thumb, a linear congruential generator is good enough for writing a game (how much damage does the monster deal) but not good enough for writing a simulation. There is a colorful history of researchers who have chosen poor PRNGs for their programs; the results of their simulations are suspect as a result.
If C++ is also acceptable for you, have a look at Boost.
http://www.boost.org/doc/libs/1_51_0/doc/html/boost_random/reference.html
It does not only offer one generator, but several dozen, and gives an overview of speed, memory requirement and randomness quality.
I am trying to optimise a C-based code used for bioinformatic purposes. It uses Monte Carlo iterations for major part of the calculations. It earlier used ran2() for generating random numbers, which was making it super slow. After thorough research, I found that mersenne twister, and sfmt are more efficient random number generators. However, I tried using them in my code, and they seem to be making not much of a difference in the speed. Given the fact that the program uses generator 10+ times for each iteration, I am not able to figure out why changing the generator is making no difference to the speed.
Could anyone tell me where am I possibly going wrong?
Choosing a random number generator is always a balance between quality (of the numbers they generate) and speed. Linear congruential generators are typically the fastest, but they are not suitable for any serious Monte Carlo work.
From experience I'd say that mersenne twister is just fine --- it's not superslow and you don't have to worry about the quality. As long as the bottleneck is in the generator, I'd say there's not much you can do on a single core.
This being said, here's a comparison of several generators:
http://www.boost.org/doc/libs/1_48_0/doc/html/boost_random/performance.html
I know that there are other questions similar to this one, however the following question pertains to arbitrary-precision random number generation in C for use in Monte Carlo simulation.
How can we generate good quality arbitrary-precision random numbers in C, when atmospheric noise isn't always available, without relying on disk i/o or network access that would create bottlenecks?
libgmp is capable of generating random numbers, but, like other implementations of pseudo-random number generators, it requires a seed. As the manual mentions, "the system time is quite easy to guess, so if unpredictability is required then it should definitely not be the only source for the seed value."
Is there a portable/ported library for generating random numbers, or seeds for random numbers? The libgmp also mentions that "On some systems there's a special device /dev/random which provides random data better suited for use as a seed." However, /dev/random and /dev/urandom can only be used on *nix systems.
Don't overestimate importance of seed.
Firstly, it doesn't need to be truly chaotic - only to have good distribution and not be correlated with any processes in your simulation or pseudo-random generator.
Secondly, for Monte-Carlo statistical characteristics matter, not randomness (in any sense) of a particular number.
Low bytes of high-precision time or some derivative of keyboard-mouse actions make a good seed for anything that is going to be run on a regular PC.
By definition, true random numbers require chaotic information from the real world. /dev/random often (but not always) provides this. Another option for *ix is Entropy Gathering Demon. /dev/urandom by design will happily provide non-random data, since it doesn't block when the entropy pool is exhausted.
Internet APIs that provide this include HotBits (radioactive decay), LavaRnd (CCD), and Random.org (atmospheric noise, which I realize you don't want). See also Hardware random number generator
This device (no affiliation) has drivers for Windows and *ix.
Why your arbitrary precision requirement? There is no "random number between 0 and infinity". You always need a range.
What's my best bet for computing the dot product of a vector x with a large number of vectors y_i, where x and y_i are of length 10k or so.
Shove the y's in a matrix and use an optimized s/dgemv routine?
Or maybe try handcoding an SSE2 solution (I don't have SSE3, according to cpuinfo).
I'm just looking for general guidance here, so any suggestions will be useful.
And yes, I do need the performance.
Thanks for any light.
I think GPUs are specifically designed to perform operations like this quickly (among others). So you could probably make use of DirectX or OpenGL libraries to perform the vector operations. D3DXVec2Dot This will also save you CPU time.
Alternatives for optimised BLAS routines:
If you use intel compilers, you may
have access to intel MKL
For other compilers ATLAS usually provides nice performance numbers
Handcoding a SSE2 solution is not very difficult and will bring a nice speedup over a pure C routine. How much this will bring over a BLAS routine must be determined by you.
The greatest speedup is derived by structuring the data into a format, so that you can exploit data parallelism and alignment.
I use a GotoBLAS. It's the hight perfomance kernel routines. The many times better than MKL and BLAS.
The following provides BLAS level 1 (vector operations) routines using SSE.
http://www.applied-mathematics.net/miniSSEL1BLAS/miniSSEL1BLAS.html
If you have an nVidia graphics card you can get cuBLAS which will perform the operation on the graphics card.
http://developer.nvidia.com/cublas
For ATI (AMD) graphics cards
http://developer.amd.com/libraries/appmathlibs/pages/default.aspx
Okay, I guess this is entirely subjective and whatnot, but I was thinking about entropy sources for random number generators. It goes that most generators are seeded with the current time, correct? Well, I was curious as to what other sources could be used to generate perfectly valid, random (The loose definition) numbers.
Would using multiple sources (Such as time + current HDD seek time [We're being fantastical here]) together create a "more random" number than a single source? What are the logical limits of the amount of sources? How much is really enough? Is the time chosen simply because it is convenient?
Excuse me if this sort of thing is not allowed, but I'm curious as to the theory behind the sources.
The Wikipedia article on Hardware random number generator's lists a couple of interesting sources for random numbers using physical properties.
My favorites:
A nuclear decay radiation source detected by a Geiger counter attached to a PC.
Photons travelling through a semi-transparent mirror. The mutually exclusive events (reflection — transmission) are detected and associated to "0" or "1" bit values respectively.
Thermal noise from a resistor, amplified to provide a random voltage source.
Avalanche noise generated from an avalanche diode. (How cool is that?)
Atmospheric noise, detected by a radio receiver attached to a PC
The problems section of the Wikipedia article also describes the fragility of a lot of these sources/sensors. Sensors almost always produce decreasingly random numbers as they age/degrade. These physical sources should be constantly checked by statistical tests which can analyze the generated data, ensuring the instruments haven't broken silently.
SGI once used photos of a lava lamp at various "glob phases" as the source for entropy, which eventually evolved into an open source random number generator called LavaRnd.
I use Random.ORG, they provide free random data from Atmospheric noise, that I use to periodically re-seed a Mersene-Twister RNG. Its about as random as you can get with no hardware dependencies.
Don't worry about a "good" seed for a random number generator. The statistical properties of the sequence do not depend on how the generator is seeded. There are other things, however. to worry about. See Pitfalls in Random Number Generation.
As for hardware random number generators, these physical sources have to be measured, and the measurement process has systematic errors. You might find "pseudo" random numbers to have higher quality than "real" random numbers.
Linux kernel uses device interrupt timing (mouse, keyboard, hard drives) to generate entropy. There's a nice article on Wikipedia on entropy.
Modern RNGs are both checked against correlations in nearby seeds and run several hundred iterations after the seeding. So, the unfortunately boring but true answer is that it really doesn't matter very much.
Generally speaking, using random physical processes have to be checked that they conform to a uniform distribution and are otherwise detrended.
In my opinion, it's often better to use a very well understood pseudo-random number generator.
I've used an encryption program that used the users mouse movement to generate random numbers. The only problem was that the program had to pause and ask the user to move the mouse around randomly for a few seconds to work properly which might not always be practical.
I found HotBits several years ago - the numbers are generated from radioactive decay, genuinely random numbers.
There are limits on how many numbers you can download a day, but it has always amused me to use these as really, really random seeds for RNG.
Some TPM (Trusted Platform Module) "chips" have a hardware RNG. Unfortunately, the (Broadcom) TPM in my Dell laptop lacks this feature, but many computers sold today come with a hardware RNG that uses truly unpredictable quantum mechanical processes. Intel has implemented the thermal noise variety.
Also, don't use the current time alone to seed an RNG for cryptographic purposes, or any application where unpredictability is important. Using a few low order bits from the time in conjunction with several other sources is probably okay.
A similar question may be useful to you.
Sorry I'm late to this discussion (what is it 3 1/2 years old now?), but I've a rekindled interest in PRN generation and alternate sources of entropy. Linux kernel developer Rusty Russell recently had a discussion on his blog on alternate sources of entropy (other than /dev/urandom).
But, I'm not all that impressed with his choices; a NIC's MAC address never changes (although it is unique from all others), and PID seems like too small a possible sample size.
I've dabbled with a Mersenne Twister (on my Linux box) which is seeded with the following algorithm. I'm asking for any comments/feedback if anyone's willing and interested:
Create an array buffer of 64 bits + 256 bits * number of /proc files below.
Place the time stamp counter (TSC) value in the first 64 bits of this buffer.
For each of the following /proc files, calculate the SHA256 sum:
/proc/meminfo
/proc/self/maps
/proc/self/smaps
/proc/interrupts
/proc/diskstats
/proc/self/stat
Place each 256-bit hash value into its own area of the array created in (1).
Create a SHA256 hash of this entire buffer. NOTE: I could (and probably should) use a different hash function completely independent of the SHA functions - this technique has been proposed as a "safeguard" against weak hash functions.
Now I have 256 bits of HOPEFULLY random (enough) entropy data to seed my Mersenne Twister. I use the above to populate the beginning of the MT Array (624 32-bit integers), and then initialize the remainder of that array with the MT author's code. Also, I could use a different hash function (e.g. SHA384, SHA512), but I'd need a different size array buffer (obviously).
The original Mersenne Twister code called for one single 32-bit seed, but I feel that's horribly inadequate. Running "merely" 2^32-1 different MTs in search of breaking the crypto is not beyond the realm of practical possibility in this day and age.
I'd love to read anyone's feedback on this. Criticism is more than welcome. I will defend my use of the /proc files as above because they're constantly changing (especially the /proc/self/* files, and the TSC always yields a different value (nanosecond [or better] resolution, IIRC). I've run Diehard tests on this (to the tune of several hundred billion bits), and it seems to be passing with flying colors. But that's probably more testament to the soundness of the Mersenne Twister as a PRNG than to how I'm seeding it.
Of course, these aren't totally impervious to someone hacking them, but I just don't see all of these (and SHA*) being hacked and broken to in my lifetime.
Some use keyboard input (timeouts between keystrokes), I heard of I think in a novel that radio static reception can be used - but of course that requires other hardware and software...
Noise on top of the Cosmic Microwave Background spectrum. Of course you must first remove some anisotropy, foreground objects, correlated detector noise, galaxy and local group velocities, polarizations etc. Many pitfalls remain.
Source of seed isn't that much important. More important is the pseudo numbers generator algorithm. However I've heard some time ago about generating seed for some bank operations. They took many factors together:
time
processor temperature
fan speed
cpu voltage
I don't remember more :)
Even if some of these parameters doesn't change much in time, you can put them into some good hashing function.
How to generate good random number?
Maybe we can take into account inifinite number of universes? If this is true, that all the time new parallel universes are being created, we can do something like this:
int Random() {
return Universe.object_id % MAX_INT;
}
In every moment we should be on another branch of parallel universes, so we should have different id. The only problem is how to get Universe object :)
How about spinning off a thread that will manipulate some variable in a tight loop for a fixed amount of time before it is killed. What you end up with will depend on the processor speed, system load, etc... Very hokey, but better than just srand(time(NULL))...
Don't worry about a "good" seed for a random number generator. The statistical properties of the sequence do not depend on how the generator is seeded.
I disagree with John D. Cook's advice. If you seed the Mersenne Twister with all bits set to zero except one, it will initially generate numbers which are anything but random. It takes a long time for the generator to churn this state into anything that would pass statistical tests. Simply setting the first 32 bits of the generator to a seed will have a similar effect. Also, if the entire state is set to zero the generator will produce endless zeroes.
Properly written RNG code will have a properly written seeding algorithm that accepts say a 64 bit value and seeds the generator so it will produce decent random numbers for each possible input. So if you are using a reliable library then any seed will do. But if you hack together your own implementation then you need to be careful.