Postgres for loop doesn't work with cursor? - database

Question

I have a simple PostgreSQL script and try to run the for loop with cursor to fetch rows as pagination:
begin;
declare curs cursor for select id from postgres.core.security_group order by id asc;
fetch 4 from curs;
commit;
Working fine, but when I add a for loop to it, won't work:
CREATE OR REPLACE FUNCTION postgres.core.cursor_selection()
RETURNS SETOF varchar AS
$func$
DECLARE
curs cursor for select * from postgres.core.security_group;
_modules varchar; -- assuming data type integer
BEGIN
FOR _modules IN SELECT * FROM postgres.core.security_group ORDER BY id limit 10
LOOP
RETURN NEXT _modules;
END LOOP;
END
$func$ LANGUAGE plpgsql;
SELECT postgres.core.cursor_selection();
I have the loop not working properly and not showing more data other than the first 10 records. How do I get the data as a set of 10s on each page?
Much appreciated.

Answer 1

Functions in plpgsql don't stream. RETURN NEXT kind of looks like it streams via co-routine or something, but it really just accumulates all the rows until the end of the function and returns them at once.
And SQL doesn't have any looping constructs. Neither does psql (in a user-visible way) that I can find. So there is no way to do it just in SQL or with psql. But in psql, you can set FETCH_COUNT, which uses a cursor and FETCH behind the scenes. (If you set log_statement=all, you can see this in action in the log file.) If you really want do it manually for some reason, you could use \watch as an infinite loop.
begin;
declare curs cursor for select id from postgres.core.security_group order by id asc;
fetch 4 from curs \watch 0.1
commit;
You will have to break out of the loop yourself once it finishes or you get tired of it, like with ctrl-C (on Linux)
The usual way to use a cursor in SQL would be in some other programming language with a db driver, like python or JS or Java or Perl. But they too will usually have settings where the driver uses cursors behind the scenes without you needing to manually implement it. (like psycopg2's "named cursors" do)