Dynamic Create Table Statement in Snowflake - snowflake-cloud-data-platform

I am trying to do the following
Creating one config table where I want to put all details which below query provides as output
USE ROLE ROLENAME;
SET SemanticLayerSchemaDetails = '''DB_D''';
SET TableSchemaDetails = 'SCHEMA_D';
SET TableNamesArray = '[''TABLE_SALES'']';
with CTE_SchemaOneObjectDetails
AS
(
SELECT
$SemanticLayerSchemaDetails AS SCHEMA_D, (''''|| TABLE_CATALOG || '''') AS DB_D,
(''''|| TABLE_NAME || '''') AS RLOBJ,listagg('"'||COLUMN_NAME||'"', ' VARCHAR , ') WITHIN GROUP (ORDER BY ORDINAL_POSITION ASC) AS COLUMN_DETAILS
from INFORMATION_SCHEMA.columns
where TABLE_SCHEMA = $TableSchemaDetails
and table_name IN (select value from table(flatten ( input => parse_json($TableNamesArray))))
GROUP BY Table_Name,TABLE_CATALOG,TABLE_SCHEMA
)
SELECT * FROM CTE_SchemaOneObjectDetails
Here , the issue is if table : TABLE_SALES has total 10 columns, for last column, I do not see VARCHAR in "COLUMN_DETAILS" field.
I want to use this "COLUMN_DETAILS" field calling from one stored procedure to create tables.
Anything we can do to tweak the code so that it adds VARCHAR in last column also?
Thank You,

Since you're using ' VARCHAR , ' as the listagg separator, it won't appear at the end. That's okay since you probably don't want the string to end with a comma anyway. You can simply concatenate ' VARCHAR' after the end of your listagg:
with CTE_SchemaOneObjectDetails
AS
(
SELECT
$SemanticLayerSchemaDetails AS SCHEMA_D, (''''|| TABLE_CATALOG || '''') AS DB_D,
(''''|| TABLE_NAME || '''') AS RLOBJ,listagg('"'||COLUMN_NAME||'"', ' VARCHAR , ') WITHIN GROUP (ORDER BY ORDINAL_POSITION ASC) || ' VARCHAR' AS COLUMN_DETAILS
from INFORMATION_SCHEMA.columns
where TABLE_SCHEMA = $TableSchemaDetails
and table_name IN (select value from table(flatten ( input => parse_json($TableNamesArray))))
GROUP BY Table_Name,TABLE_CATALOG,TABLE_SCHEMA
)
SELECT * FROM CTE_SchemaOneObjectDetails
I have a question about this though... Are all the tables you'll use this approach with defined with all varchar data types? Would you prefer to have the defined data type instead, or will this be used to create string-only versions of the table or something else that wants all columns converted to string types?
If you do want the defined data types instead of all varchar, you can use this:
with CTE_SchemaOneObjectDetails
AS
(
SELECT
$SemanticLayerSchemaDetails AS SCHEMA_D, (''''|| TABLE_CATALOG || '''') AS DB_D,
(''''|| TABLE_NAME || '''') AS RLOBJ,listagg('"'||COLUMN_NAME||'"' || ' ' || DATA_TYPE , ',') WITHIN GROUP (ORDER BY ORDINAL_POSITION ASC) AS COLUMN_DETAILS
from INFORMATION_SCHEMA.columns
where TABLE_SCHEMA = $TableSchemaDetails
and table_name IN (select value from table(flatten ( input => parse_json($TableNamesArray))))
GROUP BY Table_Name,TABLE_CATALOG,TABLE_SCHEMA
)
SELECT * FROM CTE_SchemaOneObjectDetails

Related

How to aggregate rows with "CASE WHEN" statement and build query as a string

This is my sample table:
TableName ColumnName
Sample Name
Sample MiddleName
Sample LastName
I'm trying to test the following code:
SELECT 'SELECT ' +
CASE WHEN TableName IS NOT NULL AND ColumnName IS NOT NULL THEN TableName_ColumnName
WHEN TableName IS NOT NULL AND ColumnName IS NULL THEN TableName_NULL
ELSE ISNULL(ColumnName, 'NULL')
END
+ ' FROM [TestDB].[dbo].' + TableName
FROM [TestDB].[dbo].[TestTable] WHERE TableName = 'Sample'
Here is a result I'm getting from the above, it's including the SELECT and 'FROM [TestDB].[dbo]' + TableName for each row which is not what I want:
SELECT Sample_Name FROM [TestDB].[dbo].Sample
SELECT Sample_MiddleName FROM [TestDB].[dbo].Sample
SELECT Sample_LastName FROM [TestDB].[dbo].Sample
The ideal result should look like this:
SELECT
Sample_Name
Sample_MiddleName
Sample_LastName
FROM [TestDB].[dbo].Sample
What am I missing?
If I understood correctly from your question you need a solution like this:
WITH TableNamesCTE AS
(
SELECT DISTINCT TableName
FROM TestTable
)
SELECT 'SELECT ' +
(SELECT STRING_AGG(CASE WHEN TableName IS NOT NULL AND ColumnName IS NOT NULL THEN CONCAT(TableName, '_', ColumnName)
WHEN TableName IS NOT NULL AND ColumnName IS NULL THEN CONCAT(TableName, '_NULL')
ELSE ISNULL(ColumnName, 'NULL')
END, ',')
FROM [TestTable] TT
WHERE TT.TableName = TN.TableName)
+ ' FROM [TestDB].[dbo].' + TableName
FROM TableNamesCTE TN
WHERE TN.TableName = 'Sample';
I have separated table names with CTE, so if you have 100 tables, then this query will produce 100 query strings. Once you have all tables prepared then in the second part of the query you will get aggregated all these things.
SQL Server command STRING_AGG is available from 2017 version. For more information: https://learn.microsoft.com/en-us/sql/t-sql/functions/string-agg-transact-sql?view=sql-server-ver15

Display All Columns from all Table by using Union ALL with Different no of Columns in Each Table

I have Three Tables with Different no of Columns. e.g T1(C1), T2(C1,C2,C3), T3(C1,C4). I want to generate a Dynamic SQL that will create a View like
CREATE VIEW [dbo].[vwData]
AS
SELECT C1,NULL AS C2,NULL AS C3,NULL AS C4
FROM DBO.T1
UNION ALL
SELECT C1,C2,C3,NULL AS C4
FROM DBO.T2
UNION ALL
SELECT C1,NULL AS C2,NULL AS C3,C4
FROM DBO.T3
I have achieved this goal by using two nested loop by Checking Each column If It is Existed in a table or not.
But in Production we have around 30 tables with around 60 Columns in Each table.
Create of Dynamic SQL is taking around 7 minutes and this is not Acceptable to us. We want to improve performance Further.
Immediate help would be highly appreciated.
Here's some Dynamic SQL which would create and execute what you describe. How does this compare to your current SQL's performance?
Fiddle: https://dbfiddle.uk/?rdbms=sqlserver_2017&fiddle=800747a3d832e6e29a15484665f5cc8b
declare #tablesOfInterest table(tableName sysname, sql nvarchar(max))
declare #allColumns table(columnName sysname)
declare #sql nvarchar(max)
insert #tablesOfInterest(tableName) values ('table1'), ('table2')
insert #allColumns (columnName)
select distinct c.name
from sys.columns c
where c.object_id in
(
select object_id(tableName)
from #tablesOfInterest
)
update t
set sql = 'select ' + columnSql + ' from ' + quotename(tableName)
from #tablesOfInterest t
cross apply
(
select string_agg(coalesce(quotename(c.Name), 'null') + ' ' + quotename(ac.columnName), ', ') within group (order by ac.columnName)
from #allColumns ac
left outer join sys.columns c
on c.object_id = object_id(t.tableName)
and c.Name = ac.columnName
) x(columnSql)
select #sql = string_agg(sql, ' union all ')
from #tablesOfInterest
print #sql
exec (#sql)
As mentioned in the comments, rather than running this dynamic SQL every time you need to execute this query, you could use it to generate a view which you can then reuse as required.
Adding indexes and filters to the underlying tables as appropriate could further improve performance; but without knowing more of the context, we can't give much advise on specifics.
You might try this:
I use some general tables where I know, that they share some of their columns to show the principles. Just replace the tables with your own tables:
Attention: I do not use these INFORMATION_SCHEMA tables to read their content. They serve as examples with overlapping columns...
DECLARE #statement NVARCHAR(MAX);
WITH cte(x) AS
(
SELECT
(SELECT TOP 1 * FROM INFORMATION_SCHEMA.TABLES FOR XML AUTO, ELEMENTS XSINIL,TYPE) AS [*]
,(SELECT TOP 1 * FROM INFORMATION_SCHEMA.COLUMNS FOR XML AUTO, ELEMENTS XSINIL,TYPE) AS [*]
,(SELECT TOP 1 * FROM INFORMATION_SCHEMA.ROUTINES FOR XML AUTO, ELEMENTS XSINIL,TYPE) AS [*]
--add all your tables here...
FOR XML PATH(''),TYPE
)
,AllColumns AS
(
SELECT DISTINCT a.value('local-name(.)','nvarchar(max)') AS ColumnName
FROM cte
CROSS APPLY x.nodes('/*/*') A(a)
)
,AllTables As
(
SELECT a.value('local-name(.)','nvarchar(max)') AS TableName
,a.query('*') ConnectedColumns
FROM cte
CROSS APPLY x.nodes('/*') A(a)
)
SELECT #statement=
STUFF((
(
SELECT 'UNION ALL SELECT ' +
'''' + TableName + ''' AS SourceTableName ' +
(
SELECT ',' + CASE WHEN ConnectedColumns.exist('/*[local-name()=sql:column("ColumnName")]')=1 THEN QUOTENAME(ColumnName) ELSE 'NULL' END + ' AS ' + QUOTENAME(ColumnName)
FROM AllColumns ac
FOR XML PATH('root'),TYPE
).value('.','nvarchar(max)') +
' FROM ' + REPLACE(QUOTENAME(TableName),'.','].[')
FROM AllTables
FOR XML PATH(''),TYPE).value('.','nvarchar(max)')
),1,10,'');
EXEC( #statement);
Short explanation:
The first row of each table will be tranformed into an XML. Using AUTO-mode will use the table's name in the <root> and add all columns as nested elements.
The second CTE will create a distinct list of all columns existing in any of the tables.
the third CTE will extract all Tables with their connected columns.
The final SELECT will use a nested string-concatenation to create a UNION ALL SELECT of all columns. The existance of a given name will decide, whether the column is called with its name or as NULL.
Just use PRINT to print out the #statement in order to see the resulting dynamically created SQL command.

How to get "," instead of "and" in the rows in SQL Server

I have a table Test with 1 column
Module_name
Table
Computer
Laptop
Chair
My expected output:
Table,Computer,Laptop and Chair
My Query:
declare #module_name varchar(50)
SELECT #Module_Name = COALESCE(#Module_Name + ' and ', '') + module_name FROM
(SELECT DISTINCT module_name FROM Test) T
select #module_name
I am getting the output as:
Table and Computer and Laptop and Chair
My concern is how to get the "," instead of "and".
Have you tried xml method with stuff() function ?
declare #Module_names varchar(max)
set #Module_names = stuff((select distinct ',' +Module_name
from table t
for xml path('')),1,1, '')
select REVERSE(STUFF(REVERSE(#Module_names),
CHARINDEX(',', REVERSE(#Module_names)), 1,' dna ')) as Module_names
I don't endorse this solution, like I said in the comments, "grammarisation" should be done in your presentation layer.. You can, however, achieve this in SQL like so:
Edit: Slight update to cater for a single value return.
CREATE TABLE #Sample (Module varchar(10));
INSERT INTO #Sample
VALUES ('Table'),
('Computer'),
('Laptop'),
('Chair');
GO
WITH RNs AS (
SELECT Module,
ROW_NUMBER() OVER (ORDER BY (SELECT NULL)) AS RN --SELECT NULL as there is no ID field to work with here, thus the order will be random
FROM #Sample)
SELECT STUFF((SELECT CASE WHEN RN = MAX(RN) OVER () AND RN != 1 THEN ' and ' ELSE ', ' END + Module
FROM RNs
ORDER BY RN
FOR XML PATH('')),1,2,'');
GO
DROP TABLE #Sample;
Use the following. First gather all records together with comma, then replace just the last one with "and". Will have to make sure that your column values don't contain comma or it will be misplaced with an "and" if on last occurence.
DECLARE #result VARCHAR(MAX) = STUFF(
(
SELECT DISTINCT
', ' + T.module_name
FROM
Test AS T
FOR XML
PATH('')
),
1, 2, '')
SET #result =
REVERSE(
STUFF( -- Replace
REVERSE(#result), -- ... in the reversed string
CHARINDEX(',', REVERSE(#result)), -- ... at the first position of the comma (the last one on the original string)
1, -- just 1 character (the comma)
'dna ') -- for the reversed " and"
)
SELECT #result
Used Row_number to capture last row,
CREATE TABLE test
([Module_name] varchar(8))
;
INSERT INTO test
([Module_name])
VALUES
('Table'),
('Computer'),
('Laptop'),
('Chair')
;
SELECT STUFF((SELECT CASE WHEN RN = MAX(RN) OVER () THEN ' and ' ELSE ', ' END + Module_name
from
(
SELECT Module_name,
ROW_NUMBER() OVER (ORDER BY (SELECT NULL)) AS RN
FROM test
) rns
ORDER BY RN
FOR XML PATH('')),1,2,'');

How to create comma delimited list from table with dynamic columns

I want to be able to grab all of the records in a table into a comma delimited list that I can then use to insert into a table on another database. Due to permission restrictions on the customer's server I cannot access any of the options when right-clicking on the database name, and all of the solutions I've found so far involve having permission to do so (e.g. Tasks > Export Data...)
I have tried using COALESCE to do this, however the problem is that my table could have any number of columns. Columns can be added/deleted at any time through the UI by the users and therefore I cannot hard code the columns in my select statement.
Here is what I have written so far, using a simple CTE statement where there are three columns (RowCode, RowOrd, RowText) and concatenating them into a variable that I print out. I just want to find a way to grab these column names dynamically instead of hard coding them. I'll also need to account for various types of column names by casting them each as varchar in the variable.
DECLARE #listStr VARCHAR(MAX)
;WITH tableData AS
(
SELECT *
FROM tableRows
)
SELECT
#listStr = ISNULL(#listStr + 'select ','select ') + '''' + RowCode + ''',''' + cast(RowOrd as varchar) + ''',''' + RowText + '''' + Char(13)
FROM
tableData
PRINT #listStr
The tableRows table contains the following records
RowCode RowOrd RowText
-----------------------
RowA 1 Row A
RowB 2 Row B
And the variable #listStr is currently printing this, which is correct
select 'RowA','1.00','Row A'
select 'RowB','2.00','Row B'
Thanks in advance!
With a bit of XML you can dynamically gather and "stringify" your values
Declare #tableRows table (RowCode varchar(50), RowOrd int, RowText varchar(50))
Insert Into #tableRows values
('RowA',1,'Row A'),
('RowB',2,'Row B')
Declare #listStr VARCHAR(MAX) = ''
Select #listStr = #listStr + C.String + char(13)
From #tableRows A
Cross Apply (Select XMLData = cast((Select A.* for XML RAW) as xml)) B
Cross Apply (
Select String = 'select '+Stuff((Select ',' +Value
From (
Select Value = ''''+attr.value('.','varchar(max)')+''''
From B.XMLData.nodes('/row') as A(r)
Cross Apply A.r.nodes('./#*') AS B(attr)
) X
For XML Path ('')),1,1,'')
) C
Select #listStr
Returns
select 'RowA','1','Row A'
select 'RowB','2','Row B'

Dynamic sql to convert column names with one row into table with 2 columns and several rows

After searching for several ways of converting columns to rows using PIVOT, cross join etc my question still goes unanswered
I have a Result set which returns 1 row and 147 columns
ID | Name | DOB | BloodGroup | ...... 147 columns
1 XYZ 17MAY A+ ......
My aim is to convert this result set into 2 columns and 147 rows
Column_Name | Value
ID 1
NAME XYZ
: :
How should I go about it ? I appreciate your feedback
I took the second approach Gordon mentioned in his post, but built dynamic SQL from it. I CROSS JOINED the result of a few sys table JOINs and a source table, then built a CASE statement off the column names. I UNION it all together as dynamic SQL then EXECUTE it. To make it easy, I've made all the variable items into variables which you fill out at the beginning of the routine. Here's the code:
USE AdventureWorks2012;
GO
DECLARE #MySchema VARCHAR(100),
#MyTable VARCHAR(100),
#MyUIDColumn VARCHAR(100),
#MyFieldsMaxLength VARCHAR(10),
#SQL AS VARCHAR(MAX);
SET #MySchema = 'Person';
SET #MyTable = 'Person';
-- Unique ID which routine will identify unique entities by. Will also sort on this value in the end result dataset.
SET #MyUIDColumn = 'BusinessEntityID';
-- This determines the max length of the fields you will cast in your Value column.
SET #MyFieldsMaxLength = 'MAX';
WITH cteSQL
AS
(
SELECT 1 AS Sorter, 'SELECT c.name AS ColumnName,' AS SQL
UNION ALL
SELECT 2, 'CASE' AS Statement
UNION ALL
SELECT 3, 'WHEN c.name = ''' + c.name + ''' THEN CAST(mt.' + c.name + ' AS VARCHAR(' + #MyFieldsMaxLength + ')) '
FROM sys.tables t INNER JOIN sys.columns c
ON t.object_id = c.object_id
WHERE t.name = #MyTable
UNION ALL
SELECT 4, 'END AS Value' AS Statement
UNION ALL
SELECT 5, 'FROM sys.tables t INNER JOIN sys.columns c ON t.object_id = c.object_id INNER JOIN sys.schemas s ON t.schema_id = s.schema_id, ' + #MySchema + '.' + #MyTable + ' mt WHERE t.name = ''' + #MyTable + ''' AND s.name = ''' + #MySchema + ''' ORDER BY mt. ' + #MyUIDColumn + ', c.name;'
)
SELECT #SQL =
(
SELECT SQL + ' '
FROM cteSQL
ORDER BY Sorter
FOR XML PATH ('')
);
EXEC(#SQL);
I really can't say what execution time will be like. I ran it against AdventureWorks2012, Person.Person table (~20k rows, 13 columns) on my local machine and it brought back ~2.5 million rows in about 8 seconds, if that means anything. The good thing is that its flexible to take any table seamlessly. Anyway, just thought it was a fun puzzle so decided to play with it a bit. Hope it helps.
EDIT: Thinking about it, this is probably even slower than Gordon's proposed method, but I did it aready. Oh well. (Yeah, his method works in about half the time. Getting fancy didn't help me much.)
This is called unpivot. The easiest way, conceptually, is to do:
select 'id' as column_name, cast(id as varchar(255)) as column_value
from Result
union all
select 'name', name
from Result
union all
. . .
This can be cumbersome to type. If result is a table, you can use information_schema.columns to create the SQL, something like:
select 'select ''' + column_name + ''' as column_name, cast(' + column_name + ' as varchar(255)) as column_value from result union all'
from information_schema.columns
where table_name = 'Result'
This method is not the most efficient approach, because it requires reading the table for each column. For that unpivot is the better approach. The syntax is described here.
Thanks for the response.
I figured out a way of doing it.
I got all the column names in a comma separated string variable. 2. Passed the same string to the UNPIVOT object. By this approach, hard coding of the 140 column names was completely avoided.

Resources