Cross between "dotimes" and "for" functionality? - loops

Question

I frequently find myself wanting to efficiently run a Clojure function multiple times with an integer index (like "dotimes") but also get the results out as a ready-made sequence/list (like "for").
i.e. I'd like to do something like this:
(fortimes [i 10] (* i i))
=> (0 1 4 9 16 25 36 49 64 81)
Clearly it would be possible to do:
(for [i (range 10)] (* i i))
But I'd like to avoid creating and throwing away the temporary range list if at all possible.
What's the best way to achieve this in Clojure?

Answer 1

Generating a range in a for loop, as you show in your second example, is the idiomatic solution for solving this problem in Clojure.
Since Clojure is grounded in the functional paradigm, programming in Clojure, by default, will generate temporary data structures like this. However, since both the "range" and the "for" command operate with lazy sequences, writing this code does not force the entire temporary range data structure to exist in memory at once. If used properly, there is therefore a very low memory overhead for lazy seqs as used in this example. Also, the computational overhead for your example is modest and should only grow linearly with the size of the range. This is considered an acceptable overhead for typical Clojure code.
The appropriate way to completely avoid this overhead, if the temporary range list is absolutely, positively unacceptable for your situation, is to write your code using atoms or transients: http://clojure.org/transients. It you do this, however, you will give up many of the advantages of the Clojure programming model in exchange for slightly better performance.

Answer 2

I've written an iteration macro that can do this and other types of iteration very efficiently. The package is called clj-iterate, both on github and clojars. For example:
user> (iter {for i from 0 to 10} {collect (* i i)})
(0 1 4 9 16 25 36 49 64 81 100)
This will not create a temporary list.

Answer 3

I'm not sure why you're concerned with "creating and throwing away" the lazy sequence created by the range function. The bounded iteration done by dotimes is likely more efficient, it being an inline increment and compare with each step, but you may pay an additional cost to express your own list concatenation there.
The typical Lisp solution is to prepend new elements to a list that you build as you go, then reverse that built-up list destructively to yield the return value. Other techniques to allow appending to a list in constant time are well known, but they do not always prove to be more efficient than the prepend-then-reverse approach.
In Clojure, you can use transients to get there, relying on the destructive behavior of the conj! function:
(let [r (transient [])]
(dotimes [i 10]
(conj! r (* i i))) ;; destructive
(persistent! r))
That seems to work, but the documentation on transients warns that one should not use conj! to "bash values in place"—that is, to count on destructive behavior in lieu of catching the return value. Hence, that form needs to be rewritten.
In order to rebind r above to the new value yielded by each call to conj!, we'd need to use an atom to introduce one more level of indirection. At that point, though, we're just fighting against dotimes, and it would be better to write your own form using loop and recur.
It would be nice to be able to preallocate the vector to be of the same size as the iteration bound. I don't see a way to do so.

Answer 4

(defmacro fortimes [[i end] & code]
`(let [finish# ~end]
(loop [~i 0 results# '()]
(if (< ~i finish#)
(recur (inc ~i) (cons ~#code results#))
(reverse results#)))))
example:
(fortimes [x 10] (* x x))
gives:
(0 1 4 9 16 25 36 49 64 81)

Answer 5

Hmm, can't seem to answer your comment because I wasn't registered. However, clj-iterate uses a PersistentQueue, which is part of the runtime library, but not exposed through the reader.
It's basically a list on which you can conj to the end.