Algorithm to iterate N-dimensional array in pseudo random order - arrays

Question

I have an array that I would like to iterate in random order. That is, I would like my iteration to visit each element only once in a seemingly random order.
Would it be possible to implement an iterator that would iterate elements like this without storing the order or other data in a lookup table first?
Would it be possible to do it for N-dimensional arrays where N>1?
UPDATE: Some of the answers mention how to do this by storing indices. A major point of this question is how to do it without storing indices or other data.

Answer 1

I decided to solve this, because it annoyed me to death not remembering the name of solution that I had heard before. I did however remember in the end, more on that in the bottom of this post.
My solution depends on the mathematical properties of some cleverly calculated numbers
range = array size
prime = closestPrimeAfter(range)
root = closestPrimitiveRootTo(range/2)
state = root
With this setup we can calculate the following repeatedly and it will iterate all elements of the array exactly once in a seemingly random order, after which it will loop to traverse the array in the same exact order again.
state = (state * root) % prime
I implemented and tested this in Java, so I decided to paste my code here for future reference.
import java.math.BigInteger;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Random;
public class PseudoRandomSequence {
private long state;
private final long range;
private final long root;
private final long prime;
//Debugging counter
private int dropped = 0;
public PseudoRandomSequence(int r) {
range = r;
prime = closestPrimeAfter(range);
root = modPow(generator(prime), closestPrimeTo(prime / 2), prime);
reset();
System.out.println("-- r:" + range);
System.out.println(" p:" + prime);
System.out.println(" k:" + root);
System.out.println(" s:" + state);
}
// https://en.wikipedia.org/wiki/Primitive_root_modulo_n
private static long modPow(long base, long exp, long mod) {
return BigInteger.valueOf(base).modPow(BigInteger.valueOf(exp), BigInteger.valueOf(mod)).intValue();
}
//http://e-maxx-eng.github.io/algebra/primitive-root.html
private static long generator(long p) {
ArrayList<Long> fact = new ArrayList<Long>();
long phi = p - 1, n = phi;
for (long i = 2; i * i <= n; ++i) {
if (n % i == 0) {
fact.add(i);
while (n % i == 0) {
n /= i;
}
}
}
if (n > 1) fact.add(n);
for (long res = 2; res <= p; ++res) {
boolean ok = true;
for (long i = 0; i < fact.size() && ok; ++i) {
ok &= modPow(res, phi / fact.get((int) i), p) != 1;
}
if (ok) {
return res;
}
}
return -1;
}
public long get() {
return state - 1;
}
public void advance() {
//This loop simply skips all results that overshoot the range, which should never happen if range is a prime number.
dropped--;
do {
state = (state * root) % prime;
dropped++;
} while (state > range);
}
public void reset() {
state = root;
dropped = 0;
}
private static boolean isPrime(long num) {
if (num == 2) return true;
if (num % 2 == 0) return false;
for (int i = 3; i * i <= num; i += 2) {
if (num % i == 0) return false;
}
return true;
}
private static long closestPrimeAfter(long n) {
long up;
for (up = n + 1; !isPrime(up); ++up)
;
return up;
}
private static long closestPrimeBefore(long n) {
long dn;
for (dn = n - 1; !isPrime(dn); --dn)
;
return dn;
}
private static long closestPrimeTo(long n) {
final long dn = closestPrimeBefore(n);
final long up = closestPrimeAfter(n);
return (n - dn) > (up - n) ? up : dn;
}
private static boolean test(int r, int loops) {
final int array[] = new int[r];
Arrays.fill(array, 0);
System.out.println("TESTING: array size: " + r + ", loops: " + loops + "\n");
PseudoRandomSequence prs = new PseudoRandomSequence(r);
final long ct = loops * r;
//Iterate the array 'loops' times, incrementing the value for each cell for every visit.
for (int i = 0; i < ct; ++i) {
prs.advance();
final long index = prs.get();
array[(int) index]++;
}
//Verify that each cell was visited exactly 'loops' times, confirming the validity of the sequence
for (int i = 0; i < r; ++i) {
final int c = array[i];
if (loops != c) {
System.err.println("ERROR: array element #" + i + " was " + c + " instead of " + loops + " as expected\n");
return false;
}
}
//TODO: Verify the "randomness" of the sequence
System.out.println("OK: Sequence checked out with " + prs.dropped + " drops (" + prs.dropped / loops + " per loop vs. diff " + (prs.prime - r) + ") \n");
return true;
}
//Run lots of random tests
public static void main(String[] args) {
Random r = new Random();
r.setSeed(1337);
for (int i = 0; i < 100; ++i) {
PseudoRandomSequence.test(r.nextInt(1000000) + 1, r.nextInt(9) + 1);
}
}
}
As stated in the top, about 10 minutes after spending a good part of my night actually getting a result, I DID remember where I had read about the original way of doing this. It was in a small C implementation of a 2D graphics "dissolve" effect as described in Graphics Gems vol. 1 which in turn is an adaption to 2D with some optimizations of a mechanism called "LFSR" (wikipedia article here, original dissolve.c source code here).

Answer 2

You could collect all possible indices in a list and then remove a random indece to visit. I know this is sort of like a lookup table, but i don't see any other option than this.
Here is an example for a one-dimensional array (adaption to multiple dimensions should be trivial):
class RandomIterator<T> {
T[] array;
List<Integer> remainingIndeces;
public RandomIterator(T[] array) {
this.array = array;
this.remainingIndeces = new ArrayList<>();
for(int i = 0;i<array.length;++i)
remainingIndeces.add(i);
}
public T next() {
return array[remainingIndeces.remove((int)(Math.random()*remainingIndeces.size()))];
}
public boolean hasNext() {
return !remainingIndeces.isEmpty();
}
}
On a side note: If this code is performance relevant, this method would perform worse by far, as the random removing from the list triggers copies if you use a list backed by an array (a linked-list won't help either, as indexed access is O(n)). I would suggest a lookup-structure (e.g. HashSet in Java) that stores all visited indices to circumvent this problem (though that's exactly what you did not want to use)
EDIT: Another approach is to copy said array and use a library function to shuffle it and then traverse it in linear order. If your array isn't that big, this seems like the most readable and performant option.

Answer 3

You would need to create a pseudo random number generator that generates values from 0 to X-1 and takes X iterations before repeating the cycle, where X is the product of all the dimension sizes. I don't know if there is a generic solution to doing this. Wiki article for one type of random number generator:
http://en.wikipedia.org/wiki/Linear_congruential_generator

Answer 4

Yes, it is possible. Imagine 3D array (you not likely use anything more than that). This is like a cube and where all 3 lines connect is a cell. You can enumerate your cells 1 to N using a dictionary, you can do this initialization in loops, and create a list of cells to use for random draw
Initialization
totalCells = ... (xMax * yMax * zMax)
index = 0
For (x = 0; x < xMax ; x++)
{
For (y = 0; y < yMax ; y++)
{
For (z = 0; z < zMax ; z++)
{
dict.Add(i, new Cell(x, y, z))
lst.Add(i)
i++
}
}
}
Now, all you have to do is iterate randomly
Do While (lst.Count > 0)
{
indexToVisit = rand.Next(0, lst.Count - 1)
currentCell = dict[lst[indexToVisit]]
lst.Remove(indexToVisit)
// Do something with current cell here
. . . . . .
}
This is pseudo code, since you didn't mention language you work in
Another way is to randomize 3 (or whatever number of dimensions you have) lists and then just nested loop through them - this will be random in the end.