Dependency Tracking function - database

I just wonder if anyone knows how to automatize views creation after running DROP ... CASCADE?
Now I'm trying to drop view at first with classic DROP VIEW myview statement and if I cannot drop the view because other objects still depend on it then checking out all the objects names that postgres lists and save their creates and then I run drop with cascade. Sometimes it's like over a dozen objects. But maybe you have got some idea to handle this issue in more automated way?
Maybe anybody has got some function?

Next step... (continuation of my previous answer).
function save_views(objectname text) stores views depending on objectname (view or table) in table saved_views.
function restore_views() restores views from table saved_views.
create or replace function save_views_oid(objectid oid)
returns void language plpgsql as $$
declare
r record;
begin
for r in
select distinct c.oid, c.relname, n.nspname
from pg_depend d
join pg_rewrite w on w.oid = d.objid
join pg_class c on c.oid = w.ev_class
join pg_namespace n on n.oid = c.relnamespace
where d.refclassid = 'pg_class'::regclass
and d.classid = 'pg_rewrite'::regclass
and d.refobjid = objectid
and c.oid <> objectid
loop
insert into saved_views values (
'CREATE VIEW ' || r.nspname || '.' || r.relname ||
' AS ' || pg_get_viewdef(r.oid, 'f'));
perform save_views_oid(r.oid);
end loop;
end; $$;
create or replace function save_views(objectname text)
returns void language plpgsql as $$
begin
create table if not exists saved_views(viewbody text);
truncate saved_views;
perform save_views_oid(objectname::regclass);
end; $$;
create or replace function restore_views()
returns void language plpgsql as $$
declare
viewtext text;
begin
for viewtext in
select viewbody from saved_views
loop
execute viewtext;
end loop;
drop table saved_views;
end; $$;
Test:
select save_views('my_view'); -- may be save_views('my_schema.my_view');
select * from saved_views;
Use:
select save_views('my_view');
drop view my_view cascade;
create view my_view as ...
select restore_views();

Table pg_depend contains all necessary informations, but it is not so easy to interpret them. Here you have sketch recursive function to retrieve dependencies of pg_class object in text format. You can tune up the function to your needs (and show us results:).
create or replace function dependency
(class_id regclass, obj_id regclass, obj_subid integer, dep_type "char")
returns setof text language plpgsql as $$
declare
r record;
begin
return query
select pg_describe_object(class_id, obj_id, obj_subid)
|| ' ('|| dep_type|| ')';
for r in
select classid, objid, objsubid, deptype
from pg_depend
where class_id = refclassid
and obj_id = refobjid
and (obj_subid = refobjsubid or obj_subid = 0)
loop
return query select dependency(r.classid, r.objid, r.objsubid, r.deptype);
end loop;
end; $$;
use:
select dependency('pg_class'::regclass, 'my_view'::regclass, 0, ' ');
select dependency('pg_class'::regclass, 'my_table'::regclass, 0, ' ');

Related

Transform JSON array to boolean columns in PostgreSQL

I have a column that contains a JSON array of strings, which I would like to transform into boolean columns. These columns are true if the value was present in the array.
Let's say I have the following columns in Postgres.
|"countries"|
---------------
["NL", "BE"]
["UK"]
I would like to transform this into boolean columns per market. e.g.
|"BE"|"NL"|"UK"|
--------------------
|True|True|False|
|False|False|True|
I know I can manually expand it using case statements for each country code, but there are 200+ countries.
Is there are more elegant solution?
Displaying a various list of columns whose labels are known only at the runtime is not so obvious with postgres. You need some dynamic sql code.
Here is a full dynamic solution whose result is close from your expected result and which relies on the creation of a user-defined composite type and on the standard functions jsonb_populate_record and jsonb_object_agg :
First you create the list of countries as a new composite type :
CREATE TYPE country_list AS () ;
CREATE OR REPLACE PROCEDURE country_list () LANGUAGE plpgsql AS
$$
DECLARE country_list text ;
BEGIN
SELECT string_agg(DISTINCT c.country || ' text', ',')
INTO country_list
FROM your_table
CROSS JOIN LATERAL jsonb_array_elements_text(countries) AS c(country) ;
EXECUTE 'DROP TYPE IF EXISTS country_list' ;
EXECUTE 'CREATE TYPE country_list AS (' || country_list || ')' ;
END ;
$$ ;
Then you can call the procedure country_list () just before executing the final query :
CALL country_list () ;
or even better call the procedure country_list () by trigger when the list of countries is supposed to be modified :
CREATE OR REPLACE FUNCTION your_table_insert_update()
RETURNS trigger LANGUAGE plpgsql VOLATILE AS
$$
BEGIN
IF EXISTS ( SELECT 1
FROM (SELECT jsonb_object_keys(to_jsonb(a.*)) FROM (SELECT(null :: country_list).*) AS a) AS b(key)
RIGHT JOIN jsonb_array_elements_text(NEW.countries) AS c(country)
ON c.country = b.key
WHERE b.key IS NULL
)
THEN CALL country_list () ;
END IF ;
RETURN NEW ;
END ;
$$ ;
CREATE OR REPLACE TRIGGER your_table_insert_update AFTER INSERT OR UPDATE OF countries ON your_table
FOR EACH ROW EXECUTE FUNCTION your_table_insert_update() ;
CREATE OR REPLACE FUNCTION your_table_delete()
RETURNS trigger LANGUAGE plpgsql VOLATILE AS
$$
BEGIN
CALL country_list () ;
RETURN OLD ;
END ;
$$ ;
CREATE OR REPLACE TRIGGER your_table_delete AFTER DELETE ON your_table
FOR EACH ROW EXECUTE FUNCTION your_table_delete() ;
Finally, you should get the expected result with the following query, except that the column label are lower case, and NULL is replacing false in the result :
SELECT (jsonb_populate_record(NULL :: country_list, jsonb_object_agg(lower(c.country), true))).*
FROM your_table AS t
CROSS JOIN LATERAL jsonb_array_elements_text(t.countries) AS c(country)
GROUP BY t
full test result in dbfiddle.

Combine multiple tables into one in Snowflake

Let's say I have the following monthly tables with table names formatted such that the number after the underscore refers to the month. What I want to do is to combine these 12 tables into one without having to write 10-30 insert/union all statements
table_1
table_2
table_3
table_4
table_5
table_6
table_7
table_8
table_9
table_10
table_11
table_12 -- (only 12 in this instance but could be as many as 36)
My current approach is to first create the master table with data from table_1.
create temporary table master_table_1_12 as
select * -- * to keep it simple for this example
from table_1;
Then use variables such that I can simply keep hitting the run button until it errors out with "table_13 does not exist"
set month_id=(select max(month_id) from master_table_1_12) + 1;
set table_name=concat('table_',$month_id);
insert into master_table_1_12
select *
from identifier($table_name);
Note: All monthly tables have a month_id column
Sure it saves some space on the console(compared to multiple inserts), but I still have to run it 12 times. Are Snowflake Tasks something I could use for this? I couldn't find a fitting example from their documentation to code that up but, if anyone had success with that or with a Javascript based SP for a problem like this, please enlighten.
Here's a stored procedure that will insert into master_table_1_12 from selects on table_1 through table_12. Modify as required:
create or replace procedure FILL_MASTER_TABLE()
returns string
language javascript
as
$$
var rows = 0;
for (var i=1; i<=12; i++) {
rows += insertRows(i);
}
return rows + " rows inserted into master_table_1_12.";
// End of main function
function insertRows(i) {
sql =
`insert into master_table_1_12
select *
from table_${i};`;
return doInsert(sql);
}
function doInsert(queryString) {
var out;
cmd1 = {sqlText: queryString};
stmt = snowflake.createStatement(cmd1);
var rs = stmt.execute();;
rs.next();
return rs.getColumnValue(1);
}
$$;
call fill_master_table();
By the way, if you don't have any processing to do and just need to consolidate the tables, you can do something like this:
insert into master_table_1_12
select * from table_1
union all
select * from table_2
union all
select * from table_3
union all
select * from table_4
union all
select * from table_5
union all
select * from table_6
union all
select * from table_7
union all
select * from table_8
union all
select * from table_9
union all
select * from table_10
union all
select * from table_11
union all
select * from table_12
;
Can you not create a view on top of these 12 tables. The view will be an union of all these tables.
Based on the comments below, I further elaborated my answer. please try this approach. It will provide better performance when your table is large. Partitioning it will improve performance. This is based on real experience.
CREATE TABLE SALES_2000 (REGION VARCHAR, UNITS_SOLD NUMBER);
CREATE TABLE SALES_2001 (REGION VARCHAR, UNITS_SOLD NUMBER);
CREATE TABLE SALES_2002 (REGION VARCHAR, UNITS_SOLD NUMBER);
CREATE TABLE SALES_2003 (REGION VARCHAR, UNITS_SOLD NUMBER);
INSERT INTO SALES_2000 VALUES('ASIA', 25);
INSERT INTO SALES_2001 VALUES('ASIA', 50);
INSERT INTO SALES_2002 VALUES('ASIA', 55);
INSERT INTO SALES_2003 VALUES('ASIA', 65);
CREATE VIEW ALL_SALES AS
SELECT * FROM SALES_2000
UNION
SELECT * FROM SALES_2001
UNION
SELECT * FROM SALES_2002
UNION
SELECT * FROM SALES_2003;
SELECT * FROM ALL_SALES WHERE UNITS_SOLD = 25;
I ended up creating a UDF that spits out a create view statement and a stored procedure that executes it to create a temporary view. I work with tables following specific naming convention, so you might have to tweak this solution a little for your use case. The separation of UDF and stored proc actually helps with that as you'd mostly need to tweak the SQL UDF. I am sharing a simplified version of what I actually have in the interest of keeping it representative of the tables I listed in my question.
SQL UDF FOR GENERATING A CREATE VIEW STATETEMENT
create or replace function sandbox.public.define_view(table_pattern varchar, start_month varchar, end_month varchar)
returns table ("" varchar) as
$$
with cte1(month_id) as
(select start_month::int + row_number() over (order by 1) - 1
from table(generator(rowcount=> end_month::int - start_month::int + 1)))
,cte2(month_id,statement) as
(select 0,
concat('create or replace temporary view master_',
split_part(table_pattern,'.',-1),
start_month,
'_',
end_month,
' as ')
union all
select month_id,
concat('select * from ',
table_pattern,
month_id,
case when month_id=end_month::int then ';' else ' union all ' end)
from cte1)
select listagg(statement, '\n') within group (order by month_id) as create_view_statement
from cte2
$$;
PROCEDURE FOR EXECUTING THE OUTPUT OF THE UDF ABOVE
create or replace procedure sandbox.public.create_view(TABLE_PATTERN varchar, START_MONTH varchar,END_MONTH varchar)
returns varchar not null
language Javascript
execute as caller
as
$$
sql_command = 'select * from table(sandbox.public.define_view(:1, :2, :3))';
var stmt = snowflake.createStatement({sqlText: sql_command ,binds: [TABLE_PATTERN, START_MONTH, END_MONTH]}).execute();
stmt.next();
var ddl = stmt.getColumnValue(1);
var run=snowflake.createStatement({sqlText: ddl}).execute();
run.next();
var message=run.getColumnValue(1);
return "Temporary " + message;
$$;
USAGE DEMO
set table_pattern ='sandbox.public.table_';
set start_month ='1';
set end_month = '12';
set master_view='master_'||split_part($table_pattern,'.',-1)||$start_month||'_'||$end_month;
call create_view($table_pattern, $start_month, $end_month);
select top 100 *
from identifier($master_view);

I need postgresql version of this function

I am newbie to postgresql and I need a postgresql version of this function - could you help me please?
CREATE FUNCTION [dbo].[getpersonname]
(#commid INT)
RETURNS VARCHAR(255)
AS
BEGIN
DECLARE #personnname varchar(255)
SELECT
#personnname = COALESCE(#personname + ',', '') +
ISNULL(CONVERT(VARCHAR(50), pers.personnname ),'')
FROM
dbo.comlink comli
INNER JOIN
dbo.person pers ON personid= comli_personid
WHERE
comli.comli_commid = #commid
RETURN #personnname
END
If I understand that code correctly it returns all names separated with commas.
That can easily be done with a simple SQL statement in Postgres:
create FUNCTION get_person_name(p_commid int)
RETURNS text
AS
$$
SELECT string_agg(pers.personnname, ',')
FROM dbo.comlink comli
JOIN dbo.person pers ON personid = comli_personid
WHERE comli.comli_commid= p_commid;
$$
language sql;

how to load a query into arrays

I need to load some queries into vectors to avoid temporary tables in a function:
create table mytable
( eref serial primarykey,
edia date,
eimpte numeric);
---
CREATE OR REPLACE FUNCTION peps(rseller integer)
RETURNS void AS
$BODY$
declare
dep_dia date[] := '{}';
dep_impte numeric[]:= '{}';
dep_ref integer[]:= '{}';
ndepositos integer :=0;
rec record;
begin
for rec in
select eref, edia, eimpte from mytable order by edia, eimpte
loop
ndepositos:=ndepositos+1;
dep_dia[ndepositos] :=edia;
dep_impte[ndepositos]:=eimpte;
dep_ref[ndepositos] :=eref;
end loop;
raise notice ' ndeps %', ndepositos;
end
$BODY$
language plpgsql volatile;
it does not work:
ERROR: column "edia" does not exist
LINE 1: SELECT edia
^
what am I doing wrong?
Thanks in advance
Don't loop! Postgres provides a great function for this:
SELECT array_agg(eref), array_agg(edia), array_agg(eimpte)
FROM (SELECT * from mytable order by edia, eimpte) AS foo
INTO your variables
By putting the order by in a subquery, the aggregate functions will get the values in the order you want. This should be faster than looping.
You are not having edia column while creating the table. Looking at reamining code, I feel your create table query should be like following:
create table mytable
( edia serial primarykey,
eref date,
eimpte numeric);

Stored procedure with conditional Where clause in SQL Server

I am creating a SQL Server stored procedure. It's a simple SELECT query that I am building. One of the parameters is to look for a flag parameter. If that parameter is left blank, then the SELECT should default to NOT having the WHERE clause at all.
CREATE PROCEDURE sprocA
#flagInd int
AS
BEGIN
SELECT intID, strQuestion, strAnswer, intCategory, intOrder
FROM tblA
-- Right here is where I am looking for the logic of the #flagInd to be evaluated.....
-- if #flagInd = 1 then 'WHERE flgInd=1'
-- else if #flagInd=0 then 'WHERE flgInd=0'
-- else if #flagInd IS NULL then ''
It's just a simple query and a simple thought I had, not sure if it can be done without nesting and rewriting the whole SELECT statement as part of of the IF statement.
This can be done like:
SELECT intID, strQuestion, strAnswer, intCategory, intOrder
FROM tblA
WHERE flgInd = #flgInd OR #flgInd IS NULL;
You can also use a CASE expression:
SELECT intID, strQuestion, strAnswer, intCategory, intOrder
FROM tblA
WHERE CASE
WHEN flgInd = #flgInd THEN 1
WHEN #flgInd IS NULL THEN 1
ELSE 0
END = 1;
There appears to just be a one to one mapping (from the parameter to the column) so why not use a simple where clause?
CREATE PROCEDURE sprocA
#flagInd int
AS
BEGIN
SELECT intID, strQuestion, strAnswer, intCategory, intOrder
FROM tblA WHERE flgInd = #flagInd;

Resources