I am trying to write a simple SQL query in pgAdmin to LOOP through each table in a database and change a specified column name IF it exists. I have never coded in SQL but after searching through many forums have managed to come up with:
DO
BEGIN
FOR i IN SELECT table_name FROM information_schema.tables
LOOP
IF SELECT column_name FROM information_schema.columns WHERE table_name = 'i.table_name'
THEN
ALTER TABLE i.table_name RENAME COLUMN old_column_name TO new_column_name
END IF;
END LOOP;
You can skip information_schema.tables entirely. Just:
DO
$$
DECLARE
rec record;
BEGIN
FOR rec IN
SELECT table_schema, table_name, column_name
FROM information_schema.columns
WHERE column_name = 'x'
LOOP
EXECUTE format('ALTER TABLE %I.%I RENAME COLUMN %I TO newname;',
rec.table_schema, rec.table_name, rec.column_name);
END LOOP;
END;
$$
LANGUAGE plpgsql;
with appropriate substitutions for 'x' and newname. Or make it into a function that takes them as parameters, whatever.
It doesn't seem like you need to query tables, just query columns for the old column name, and get the table it's in.
You will need to use Dynamic SQL to do the renames.
See here on how to loop through query results. This of course needs to be done in a PL/pgSQL function.
The correct answer for your question was posted above. Alternately if you are trying to simply drop a column from all your tables as I was trying to do, then you can do it this way
DO
$$
DECLARE
rec record;
BEGIN
FOR rec IN
SELECT table_schema, table_name, column_name
FROM information_schema.columns
WHERE column_name = 'my_column_name_to_drop'
LOOP
EXECUTE format('ALTER TABLE %I.%I DROP %I;',
rec.table_schema, rec.table_name, rec.column_name);
END LOOP;
END;
$$
LANGUAGE plpgsql;
Where my_column_name_to_drop is the actual column you are searching for.
Related
The ask is to get the definition of all the views from Production environment and refresh the lower environment. I assume, GET_DDL in a loop will suffice the need but not sure how to implement it. Please advise.
Assuming that VIEW_DEFITION is not enough:
SELECT TABLE_CATALOG, TABLE_SCHEMA, TABLE_NAME, VIEW_DEFINITION
FROM INFORMATION_SCHEMA.VIEWS
WHERE TABLE_SCHEMA != 'INFORMATION_SCHEMA';
and GET_DDL is required:
DECLARE
CUR CURSOR FOR SELECT CONCAT_WS('.',TABLE_CATALOG,TABLE_SCHEMA,TABLE_NAME) AS name
FROM INFORMATION_SCHEMA.VIEWS
WHERE TABLE_SCHEMA NOT IN ('INFORMATION_SCHEMA');
BEGIN
CREATE OR REPLACE TEMPORARY TABLE temp_view_defs(view_name TEXT, definition TEXT);
FOR rec IN CUR DO
EXECUTE IMMEDIATE REPLACE('INSERT INTO temp_view_defs(view_name, definition)
SELECT ''<view_name>'', GET_DDL(''TABLE'', ''<view_name>'')'
,'<view_name>'
,rec.name);
END FOR;
LET rs RESULTSET := (SELECT * FROM temp_view_defs);
RETURN TABLE(rs);
END;
Sample output:
Check this. You can download the results and use it to get the DDL of all views, at once.
SELECT 'SELECT GET_DDL(''VIEW'',''' || table_name || ''');' AS stmt
FROM INFORMATION_SCHEMA.views
WHERE table_name NOT IN('TABLES','COLUMNS','SCHEMATA','SEQUENCES','VIEWS','TABLE_PRIVILEGES','USAGE_PRIVILEGES','DATABASES','REPLICATION_DATABASES','REPLICATION_GROUPS','FUNCTIONS','PROCEDURES','OBJECT_PRIVILEGES','OBJECT_PRIVILEGES','FILE_FORMATS','APPLICABLE_ROLES','ENABLED_ROLES','STAGES','REFERENTIAL_CONSTRAINTS','TABLE_CONSTRAINTS','INFORMATION_SCHEMA_CATALOG_NAME','LOAD_HISTORY','TABLE_STORAGE_METRICS','PIPES','EXTERNAL_TABLES','LOGGERS','EVENT_TABLES','PACKAGES');
Thanks all for suggesting your solutions. I found below code is much more close to my requirement; copy entire VIEWS in one go:
select view_definition from information_schema.views
where table_schema = 'XYZ'
Copy the the view_definition > Execute it.
Let's assume we have a table that contains data as below:
CREATE TABLE tab(i INT PRIMARY KEY);
INSERT INTO tab(i) VALUES(1),(2),(3);
SELECT * FROM tab;
Now my goal is to create SQL script that will add a new column to existing table:
ALTER TABLE IF EXISTS tab ADD COLUMN col VARCHAR(10);
Everything works as intended. Except the fact I would like to be able to run script multiple times but the effect should take place only once(idempotence).
If I try to run it again I will get:
SQL compilation error: column COL already exists
Normally I would use one of these approaches:
a) Using control structure IF to check metadata tables before executing query:
-- (T-SQL)
IF NOT EXISTS(SELECT * FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME='TAB' AND COLUMN_NAME = 'COL')
BEGIN
ALTER TABLE tab ADD col VARCHAR(10);
END;
db<>fiddle demo
I have not found IF statement in Snowflake's documentation.
b) SQL dialect that supports IF NOT EXISTS syntax:
-- PostgreSQL
ALTER TABLE IF EXISTS tab ADD COLUMN IF NOT EXISTS col VARCHAR(10);
db<>fiddle demo
Most of Snowflake SQL commands contain IF EXISTS/OR REPLACE clauses which means it was written in a way to allow running scripts multiple times.
I was considering using code like:
CREATE OR REPLACE TABLE tab
AS
SELECT i, CAST(NULL AS VARCHAR(10)) AS col
FROM tab;
This approach on other hand causes unnecessary table creation and does not preserve metadata(like primary key).
Is there a way to achieve similar effect on Snowflake? Preferably by using conditional code(add column is an example).
You can use something like this. It will report the failure to add the column if it already exists, but it will handle the error so it won't interfere with the execution of a sql script:
create or replace procedure SafeAddColumn(tableName string, columnName string, columnType string)
returns string
language JavaScript
as
$$
var sql_command = "ALTER TABLE IF EXISTS " + TABLENAME + " ADD COLUMN " + COLUMNNAME + " " + COLUMNTYPE + ";";
var strOut;
try {
var stmt = snowflake.createStatement( {sqlText: sql_command} );
var resultSet = stmt.execute();
while (resultSet.next()) {
strOut = resultSet.getColumnValue(1);
}
}
catch (err) {
strOut = "Failed: " + err; // Return a success/error indicator.
}
return strOut;
$$;
CREATE OR REPLACE TABLE tab(i INT PRIMARY KEY);
INSERT INTO tab(i) VALUES(1),(2),(3);
SELECT * FROM tab;
call SafeAddColumn('tab', 'col', 'varchar(10)');
select * from tab;
call SafeAddColumn('tab', 'col', 'varchar(10)');
It is possible to write conditional code using Snowflake Scripting.
Working with Branching Constructs
Snowflake Scripting supports the following branching constructs:
IF-THEN-ELSEIF-ELSE
CASE
Setup:
CREATE OR REPLACE TABLE PUBLIC.tab(i INT PRIMARY KEY);
INSERT INTO tab(i) VALUES(1),(2);
SELECT * FROM tab;
-- i
-- 1
-- 2
Code that can be rerun multiple times(subsequent runs will take no effect):
-- Snowsight
BEGIN
IF (NOT EXISTS(SELECT *
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'TAB'
AND TABLE_SCHEMA = 'PUBLIC'
AND COLUMN_NAME = 'COL')) THEN
ALTER TABLE IF EXISTS tab ADD COLUMN col VARCHAR(10);
END IF;
END;
EXECUTE IMMEDIATE is required is run using "classic web interface":
EXECUTE IMMEDIATE $$
BEGIN
IF (NOT EXISTS(SELECT *
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'TAB'
AND TABLE_SCHEMA = 'PUBLIC'
AND COLUMN_NAME = 'COL')) THEN
ALTER TABLE IF EXISTS tab ADD COLUMN col VARCHAR(10);
END IF;
END;
$$
After:
SELECT * FROM tab;
-- i col
-- 1 NULL
-- 2 NULL
Although Snowflake has implemented a pretty rich mix of DDL and DML for their SQL implementation, when it comes to procedural code they seem to be relying on JavaScript, at least at this point. But you should be able to accomplish your idempotent ALTER script through a JavaScript stored procedure.
I'm afraid I lack the JavaScript skills to provide you with a working sample myself at this point. The organization I'm with recently adopted Snowflake, though, so I'll share some of my research.
Here's a recent blog post on just this question:
Snowflake Control Structures – IF, DO, WHILE, FOR
Snowflake's overview documentation regarding stored procedures:
Stored Procedures
On the page above, what is currently the third link down contains extensive sample code.
Working With Stored Procedures
Building on Lukasz answer, to include database in condition you can use:
execute immediate $$
BEGIN
IF (
NOT EXISTS(
SELECT *
FROM "INFORMATION_SCHEMA"."COLUMNS"
WHERE
"TABLE_CATALOG" = 'DB_NAME'
AND "TABLE_SCHEMA" = 'SCHEMA_NAME'
AND "TABLE_NAME" = 'TABLE_NAME'
AND "COLUMN_NAME" = 'col_name'
)
) THEN
ALTER TABLE IF EXISTS "DB_NAME"."SCHEMA_NAME"."TABLE_NAME"
ADD COLUMN "col_name" VARCHAR NULL;
END IF;
END;
$$;
I have a trigger in mssql in which I want to compare each column from the inserted table with the deleted table to check if the value has changed...
If the value has changed I want to insert the column name into a temp table.
My code until now:
declare columnCursor CURSOR FOR
SELECT COLUMN_NAME
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'MyTable' AND TABLE_SCHEMA='dbo'
--save inserted and deleted into temp tables
select * into #row1 from Inserted
select * into #row2 from Deleted
declare #tmp table(column_name nvarchar(max))
declare #column nvarchar(50)
OPEN COlumnCUrsor
FETCH NEXT FROM ColumnCursor INTO #column
while ##FETCH_STATUS=0 begin
declare #out bit
declare #sql nvarchar(max) = N'
select #out = case when r1.'+#column+'r2.'+#column+' then 1 else 0 end
from #row1 r1
left join #row2 r2 on r1.sys_volgnr=r2.sys_volgnr'
exec sp_executesql #sql,N'#out bit OUTPUT', #out=#out OUTPUT
if( #out = 1 ) begin
insert into #tmp VALUES(#column)
end
FETCH NEXT FROM ColumnCursor INTO #column
end
CLOSE ColumnCursor;
DEALLOCATE ColumnCursor;
Is there an easier way to accomplish this?
Yes, there is.
You can use the COLUMNS_UPDATED function to determine the columns that had actually changed values, though it's not a very friendly function in terms of code readability.
Read this article from Microsoft support called Proper Use of the COLUMNS_UPDATED() Function to see what I mean.
I've came across an article called A More Performant Alternative To COLUMNS_UPDATED(), perhaps it can help you or at least inspire you.
I will note that you should resist the temptation to use the UPDATE() function, as it may return true even if no data was changed.
here is the relevant part from it's MSDN page:
UPDATE() returns TRUE regardless of whether an INSERT or UPDATE attempt is successful.
Looks like you're trying to build a dynamic solution, which might be useful if you expect to change often (=new columns to be added etc). You could do something like this (in pseudo-code)
Build a dynamic SQL based on DMVs (INFORMATION_SCHEMA.COLUMNS) for the column names:
insert into table ...
select
function_to_split_by_comma (
case when I.col1 = U.col1 then 'col1,' else '' end +
case when I.col2 = U.col2 then 'col2,' else '' end +
...
)
where
I.key_column1 = U.key_column1 ...
These names (col1, col2) should be the columns from the DMV query, + the case for each of the row, and then fixed SQL part for the beginning + you'll need to figure out how to join inserted and deleted, which requires the primary key.
For splitting the data into rows, you can use for example the delimited_split_8k by Jeff Moden (http://www.sqlservercentral.com/articles/Tally+Table/72993/).
Also as Damien pointed out, there can be more than one row in the inserted / deleted tables.
I have a trigger, but I need to associate with all tables of the my postgres.
Is there a command like this below?
CREATE TRIGGER delete_data_alldb
BEFORE DELETE
ON ALL DATABASE
FOR EACH ROW
EXECUTE PROCEDURE delete_data();
Well there is no database-wide trigger creation but for all such bulk-admin-operations you could use PostgreSQL system tables to generate queries for you instead of writing them by hand.
In this case you could run:
SELECT
'CREATE TRIGGER '
|| tab_name
|| ' BEFORE DELETE ON ALL DATABASE FOR EACH ROW EXECUTE PROCEDURE delete_data();' AS trigger_creation_query
FROM (
SELECT
quote_ident(table_schema) || '.' || quote_ident(table_name) as tab_name
FROM
information_schema.tables
WHERE
table_schema NOT IN ('pg_catalog', 'information_schema')
AND table_schema NOT LIKE 'pg_toast%'
) tablist;
This will get you set of strings which are SQL commands like:
CREATE TRIGGER schema1.table1 BEFORE DELETE ON ALL DATABASE FOR EACH ROW EXECUTE PROCEDURE delete_data();
CREATE TRIGGER schema1.table2 BEFORE DELETE ON ALL DATABASE FOR EACH ROW EXECUTE PROCEDURE delete_data();
CREATE TRIGGER schema1.table3 BEFORE DELETE ON ALL DATABASE FOR EACH ROW EXECUTE PROCEDURE delete_data();
CREATE TRIGGER schema2.table1 BEFORE DELETE ON ALL DATABASE FOR EACH ROW EXECUTE PROCEDURE delete_data();
CREATE TRIGGER schema2."TABLE2" BEFORE DELETE ON ALL DATABASE FOR EACH ROW EXECUTE PROCEDURE delete_data();
...
etc
You just need to run them at once (either by psql or pgAdmin).
Now some explanation:
I select names of tables in my database using information_schema.tables system table. Because there are data of literally all tables, remember to exclude pg_catalog and information_schema schemas and toast tables from your select.
I use quote_ident(text) function which will put string inside double quote signs ("") if necessary (ie. names with spaces or capital letters require that).
When I have list of tables names I just concatenate them with some static strings to get my SQL commands.
I write that command using sub-query because I want you to get better idea of what's going on here. You may write a single query by putting quote_ident(table_schema) || '.' || quote_ident(table_name) in place of tab_name.
A conveniently encapsulated version of Gabriel's answer. This time I am using the trigger to update a column named update_dt datetime granted to be part of any table in the public schema of the current database.
--
-- function: tg_any_update_datetime_fn
-- when: before insert or update
--
create or replace function tg_any_update_datetime_fn ()
returns trigger
language plpgsql as $$
begin
new.update_dt = now();
return new;
end;
$$;
--
-- function: ddl_create_before_update_trigger_on_all_tables
-- returns: Create a before update trigger on all tables.
--
create or replace procedure ddl_create_before_update_trigger_on_all_tables ()
language plpgsql as $$
declare
_sql varchar;
begin
for _sql in select concat (
'create trigger tg_',
quote_ident(table_name),
'_before_update before update on ',
quote_ident(table_name),
' for each row execute procedure tg_any_update_datetime_fn ();'
)
from
information_schema.tables
where
table_schema not in ('pg_catalog', 'information_schema') and
table_schema not like 'pg_toast%'
loop
execute _sql;
end loop;
end;
$$;
-- create before update trigger on all tables
call ddl_create_before_update_trigger_on_all_tables();
On my DDL scripts I use a large number of such ddl_ functions that have only meaning at DDL time. To remove them from the database use
--
-- function: ddl_drop_ddl_functions
-- returns: Drop all DDL functions.
-- since: 1.1.20
--
create or replace procedure ddl_drop_ddl_functions ()
language plpgsql as $$
declare
r record;
_sql varchar;
begin
for r in
select oid, prokind, proname
from pg_proc
where pronamespace = 'public'::regnamespace
and proname ilike 'ddl_%'
loop
case r.prokind
when 'a' then _sql = 'aggregate';
when 'p' then _sql = 'procedure';
else _sql = 'function';
end case;
_sql = format('drop %s %s', _sql, r.oid::regprocedure);
execute _sql;
end loop;
end
$$;
I am trying to write this query to find all tables with specific column with some specific value. This is what I've done so far -
EXEC sp_MSforeachtable
#command1='
IF EXISTS (SELECT * FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA=PARSENAME("?",2) AND TABLE_NAME=PARSENAME("?",1) AND COLUMN_NAME="EMP_CODE")
BEGIN
IF (SELECT COUNT(*) FROM ? WHERE EMP_CODE="HO081")>0
BEGIN
SELECT * FROM ? WHERE EMP_CODE="HO081"
END
END
'
I hope my intensions are clear, I just want to select only those tables where the column EMP_CODE is present and in those tables I want to select those rows where EMP_CODE='HO081'.
Edit -
Now it stands like this. But I'm not able to replace #EMPCODE variable in the query.
DECLARE #EMPCODE AS VARCHAR(20)
SET #EMPCODE='HO081'
EXEC sp_MSforeachtable
#command1='
DECLARE #COUNT AS INT
SELECT #COUNT=COUNT(*) FROM ? WHERE EMP_CODE='''+#EMPCODE+'''
IF #COUNT>0
BEGIN
PRINT PARSENAME("?",1)+'' => ''+CONVERT(VARCHAR,#COUNT)+'' ROW(S)''
--PRINT ''DELETE FROM ''+PARSENAME("?",1)+'' WHERE EMP_CODE='''''+#EMPCODE+'''''''
END
',#whereand='AND O.ID IN (SELECT OBJECT_ID FROM SYS.COLUMNS C WHERE C.NAME='''+#EMPCODE+''')'
You know how sp_MSforeachtable is undocumented, and may go away at any time/be modified?
Well, if you're happy to ignore that, it has another parameter called #whereand, which is appended to the WHERE clause of the internal query that is being used to find the tables (and should start with an AND).
You also have to know that there's an alias, o against sysobjects, and a second alias syso against sys.all_objects.
Using this knowledge, you might craft your #whereand parameter as:
EXEC sp_MSforeachtable
#command1='...',
#whereand='AND o.id in (select object_id from sys.columns c where c.name=''EMP_CODE'')'
You can now also simplify your command1, since you know it will only be run against tables containing an EMP_CODE column. I'd probably take out the COUNT(*) condition also, since I don't see what value it's adding.
Updated based on your further work, and tested against one table:
DECLARE #EMPCODE AS VARCHAR(20)
SET #EMPCODE='HO081'
declare #sql nvarchar(2000)
set #sql = '
DECLARE #COUNT AS INT
SELECT #COUNT=COUNT(*) FROM ? WHERE EMP_CODE='''+#EMPCODE+'''
IF #COUNT>0
BEGIN
PRINT PARSENAME("?",1)+'' => ''+CONVERT(VARCHAR,#COUNT)+'' ROW(S)''
--PRINT ''DELETE FROM ''+PARSENAME("?",1)+'' WHERE EMP_CODE='''''+#EMPCODE+'''''''
END
'
EXEC sp_MSforeachtable
#command1=#sql,#whereand='AND O.ID IN (SELECT OBJECT_ID FROM SYS.COLUMNS C WHERE C.NAME=''EMP_CODE'')'
(I've reverted the #whereand to query for EMP_CODE, since you don't want to replace the value there).
The issue is that, you can pass parameters to a stored procedure, or literals, but you can't perform calculations/combining actions between them - so I moved the construction of the sql statement out into a separate action.
I guess you get an error of some kind, perhaps Invalid column name 'EMP_CODE'?
It's because the code is compiled before you check for the column.
You could do like this instead.
EXEC sp_MSforeachtable
#command1='
IF EXISTS (SELECT * FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA=PARSENAME("?",2) AND TABLE_NAME=PARSENAME("?",1) AND COLUMN_NAME="EMP_CODE")
BEGIN
EXEC(''
IF (SELECT COUNT(*) FROM ? WHERE EMP_CODE="HO081")>0
BEGIN
SELECT * FROM ? WHERE EMP_CODE="HO081"
END
'')
END
'