Lua How iterators and for loops are work, what is it? - loops

Question

How does it work and why does the loop return values for k, v vars in order?
Why if I just call the next() function many times it does not return the same result as in the loop?
Where does the for loop get the values for the second argument of the next() function?
I don't understand how it works
local t = {'a', 'b', 'c'}
-- prints 1 a, 2 b, 3 c
for k, v in next, t, nil do
print(k, v)
end
print()
print(next(t)) -- 1 a
print(next(t)) -- why not 2 b?
print(next(t)) -- why not 3 c?
I don't understand much there https://www.lua.org/pil/7.html, I asked a question. How it works in lua - iterators, for loops, closures

Answer 1

next is a stateless iterator. That is, next is a pure function - its return values are determined only by its inputs (the invariant state: the table & the loop control variable: the key)
This explains why
print(next(t)) -- 1 a
print(next(t)) -- why not 2 b?
print(next(t)) -- why not 3 c?
must print 1 a three times: Each time, you call next with the table as first (and nothing as second argument). If the second argument to next is nil or nothing, next returns the first key-value pair of the given table. To get the next k-v-pair, you need to pass that key to next: next(t, 1) may give you 2 b. To fix your subsequent calls to next, you need to always pass the last key to get the next k-v-pair:
local k, v = next(t)
print(k, v) -- 1st entry
k, v = next(t, k)
print(next(t, k)) -- 2nd entry
k, v = next(t, k)
print(next(t, k)) -- 3rd entry
note that it is not guaranteed that next traverses the list part of a table in order (although all implementations do it this way)
As outlined in the PIL chapter 7.2, a for-loop is just syntactic sugar to iterate over such an iterator (for the sake of simplicity, we assume every iterator returns two values like next):
for k, v in next, t, nil do
print(k, v)
end
is equivalent to
local iterator_func, invariant_state, control_var = next, t, nil
while true do
local k, v = iterator_func(invariant_state, control_var)
if k == nil then break end
print(k, v)
control_var = k
end
pairs(t) is just even more sugar for next, t, nil (which may also be written as next, t). You can trivially implement pairs yourself as function pairs() return next, t, nil end. ipairs works similarly, except it doesn't use next but an "inext" iterator which only considers integer keys (and guarantees inorder traversal).
Stateful iterators on the other hand usually have a hidden state (usually upvalues of a closure). Every call to the iterator function changes the hidden state; the invariant state and loop control variable usually is not needed at all (and ignored entirely). These iterators behave as you would expect. We can write a stateful pairs (that is, a pairs which always returns a closure remembering the current position of the "cursor" in the table) as follows by making the current key an upvalue:
function stateful_pairs(t)
local key
-- This is the iterator function: Note how we may take no params
-- since we have everything as an upvalue already
return function()
local value
key, value = next(t, key)
return key, value
end
end
this now works the way you expected in your example, because the closure returned by this pairs remembers its state in upvalues:
local stateful_next = stateful_pairs(t)
print(stateful_next(t)) -- 1 a
print(stateful_next(t)) -- 2 b
print(stateful_next(t)) -- 3 c
and you can still use this in a for-loop. For stateful iterators without parameters, a for loop is just syntactic sugar for calling a function until it returns nil:
for k, v in stateful_pairs(t) do
print(k, v)
end
Lua has a few stateful iterators such as io.lines or string.gmatch in its standard library; making them stateless would require a very ugly loop control variable & invariant state.