Related
What i am trying to accomplish is comparing two rows to each other pointing out the differences from row to row. Each row has quite a few columns and I was trying to make it easily visible for which ones had changed. Code below is my thoughts, but I know this won't work, but is a start.
SELECT
(SELECT concat('Case WHEN T1.', column_name, ' <> T2.', column_name, ' THEN ''', column_name, ' Changed Values('' + CONVERT(varchar(100), T1.', column_name, ') + '', '' + CONVERT(varchar(100), T2.', column_name, ') + '')'' ELSE '''' END AS ', column_name)
FROM information_schema.columns
WHERE table_name = 'Table')
FROM
(
SELECT * FROM Table
WHERE ID = '13'
) AS T1
JOIN
(
SELECT * FROM Table
WHERE ID = '2006'
) AS T2
ON T1.CreateTimeStamp = T2.CreateTimeStamp
I got the idea because below this works fine, but I would like this to be potentially reusable code for other table without having to type out tens or hundreds of columns each time.
SELECT
Case WHEN T1.R1<> T2.R1 THEN 'Changed Values(' + CONVERT(varchar(100),T1.R1) + ', ' + CONVERT(varchar(100),T2.R1) + ')' ELSE '' END AS R1,
Case WHEN T1.R2<> T2.R2 THEN 'Changed Values(' + CONVERT(varchar(100),T1.R2) + ', ' + CONVERT(varchar(100),T2.R2) + ')' ELSE '' END AS R2
FROM
(
SELECT * FROM Table
WHERE ID = '13'
) AS T1
JOIN
(
SELECT * FROM Table
WHERE ID = '2006'
) AS T2
ON T1.CreateTimeStamp = T2.CreateTimeStamp
For the this example please assume CreateTimeStamp always equals each other between the two rows.
You would need to create the whole query as dynamic SQL. Note that I'm using QUOTENAME() to prevent SQL Injection from weirdly named columns. I'm also trying to keep a format for the code, so I won't get headaches when debugging.
DECLARE #SQL NVARCHAR(MAX);
SELECT #SQL = N' SELECT ' + NCHAR(10)
--Concatenate all columns except ID and CreateTimeStamp
+ STUFF(( SELECT REPLACE( CHAR(9) + ',CASE WHEN T1.<<ColumnName>> <> T2.<<ColumnName>> ' + CHAR(10)
+ CHAR(9) + CHAR(9) + 'THEN ''Changed Values('' + CONVERT(varchar(100),T1.<<ColumnName>>) + '', '' + CONVERT(varchar(100),T2.<<ColumnName>>) + '')'' ' + CHAR(10)
+ CHAR(9) + CHAR(9) + 'ELSE '''' END AS <<ColumnName>>', '<<ColumnName>>', QUOTENAME(COLUMN_NAME)) + NCHAR(10)
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'Table'
AND COLUMN_NAME NOT IN( 'ID', 'CreateTimeStamp')
FOR XML PATH(''), TYPE).value('./text()[1]', 'nvarchar(max)'), 2, 1, '') + NCHAR(10)
--Add rest of the query
+ 'FROM Table AS T1 ' + NCHAR(10)
+ 'JOIN Table AS T2 ON T1.CreateTimeStamp = T2.CreateTimeStamp ' + NCHAR(10)
+ 'WHERE ID = #ID1 ' + NCHAR(10)
+ 'AND ID = #ID2;'
--PRINT for debugging purposes
PRINT #SQL;
--Execute the dynamic built code
EXECUTE sp_executesql #SQL,
N'#ID1 int, #ID2 int',
#ID1 = 13,
#ID2 = 2006;
The concatenation method is explained on this article.
I have built a stored procedure that aims to identify duplicates in a table and to display the duplicated rows in a meaningful order. It looks like this:
CREATE PROCEDURE [dbo].[spFindDuplicates]
#tableName nvarchar(255),
#field1 nvarchar(255),
#field2 nvarchar(255) = '1',
#field3 nvarchar(255) = '2',
#field4 nvarchar(255) = '3',
#field5 nvarchar(255) = '4'
AS
BEGIN
DECLARE #query AS nvarchar(MAX);
SET #query = '
SELECT *
FROM ' + #tableName + '
WHERE CAST(' + #field1 + ' AS nvarchar(255)) + CAST(' + #field2 + ' AS nvarchar(255)) + CAST(' + #field3 + ' AS nvarchar(255)) + CAST(' + #field4 + ' AS nvarchar(255)) + CAST(' + #field5 + ' AS nvarchar(255))
IN
(
SELECT CAST(' + #field1 + ' AS nvarchar(255)) + CAST(' + #field2 + ' AS nvarchar(255)) + CAST(' + #field3 + ' AS nvarchar(255)) + CAST(' + #field4 + ' AS nvarchar(255)) + CAST(' + #field5 + ' AS nvarchar(255))
FROM ' + #tableName + '
GROUP BY CAST(' + #field1 + ' AS nvarchar(255)) + CAST(' + #field2 + ' AS nvarchar(255)) + CAST(' + #field3 + ' AS nvarchar(255)) + CAST(' + #field4 + ' AS nvarchar(255)) + CAST(' + #field5 + ' AS nvarchar(255))
HAVING COUNT(*) > 1
)
ORDER BY ' + #field1 + ', ' + #field2 + ', ' + #field3 + ', ' + #field4 + ', ' + #field5
EXECUTE(#query);
END
GO
--Example:
EXEC spFindDuplicates #tableName = 'someRandomTable', #field1 = 'firstField', #field2 = 'secondField', #field3 = 'thirdField'
As you can see, I can use at most 5 different fields that I concatenate in order for me to get a key used to determine whether we have a duplicate or not. Please note that I use the CAST function to be able to concatenate fields with various datatypes (varchar, int, dates, etc.).
When I execute the above stored procedure with 5 different fields, it works fine. But I would like to be able to run it with a variable number of fields (from 1 to 5), which is why I provided default values for #field2 to #field5.
But when I execute it with the above example (3 fields provided), I get the following error message:
A column has been specified more than once in the order by list. Columns in the order by list must be unique.
QUESTION: How can I keep ordering the resulting table without getting an error?
BONUS QUESTION: If you find a dynamic way to use that stored procedure with any number of fields (4, 17, or whatever), that'd be even more useful to me.
Like I said in the comments, injection is a huge problem here, and you need to consider it. Saying "Let's consider I don't mind about injection" is naïve and you need to change that attitude. Always make your SQL safe; then there are no excuses and chances for your application being compromised.
As what you are after, I suspect this achieves the goal. There's no need for the subquery to scan your table with an IN here, you can make use of COUNT and the OVER clause within a CTE.
CREATE PROCEDURE [dbo].[FindDuplicates] --I've removed te sp prefix, as sp_ is reserved by MS
#tableName sysname,
#field1 sysname,
#field2 sysname = NULL,
#field3 sysname = NULL,
#field4 sysname = NULL,
#field5 sysname = NULL
AS BEGIN
DECLARE #query AS nvarchar(MAX);
SET #query = N'WITH CTE AS(' + NCHAR(10) +
N' SELECT *' + NCHAR(10) +
N' COUNT(*) OVER (PARTITION BY ' + STUFF(CONCAT(N',' + QUOTENAME(#field1),N',' + QUOTENAME(#field2),N',' + QUOTENAME(#field3),N',' + QUOTENAME(#field4),N',' + QUOTENAME(#field5)),1,1,N'') + N' AS RowCount' + NCHAR(10) +
N' FROM ' + QUOTENAME(#tableName) + N')' + NCHAR(10) +
N'SELECT *' + NCHAR(10) +
N'FROM CTE' + NCHAR(10) +
N'WHERE RowCount > 1' + NCHAR(10) +
N'ORDER BY ' + STUFF(CONCAT(N',' + QUOTENAME(#field1),N',' + QUOTENAME(#field2),N',' + QUOTENAME(#field3),N',' + QUOTENAME(#field4),N',' + QUOTENAME(#field5)),1,1,N'') + N';';
PRINT #query;
--EXEC sys.sp_executesql #query; --Uncomment to rrun the actual query
END
GO
For the command you gave us EXEC dbo.FindDuplicates #tableName = 'someRandomTable', #field1 = 'firstField', #field2 = 'secondField', #field3 = 'thirdField';, this returns the SQL:
WITH CTE AS(
SELECT *
COUNT(*) OVER (PARTITION BY [firstField],[secondField],[thirdField] AS RowCount
FROM [someRandomTable])
SELECT *
FROM CTE
WHERE RowCount > 1
ORDER BY [firstField],[secondField],[thirdField];
Which, I believe gives you the behaviour you are after.
Edited the code to check if the column list exists on the sys.columns there by making sure we get only the columns which are appropriate.
CREATE FUNCTION dbo.fn_SplitString
(
#List NVARCHAR(MAX),
#Delimiter NVARCHAR(255)
)
RETURNS TABLE
WITH SCHEMABINDING
AS
RETURN
(
SELECT Item = y.i.value('(./text())[1]', 'nvarchar(4000)')
FROM
(
SELECT x = CONVERT(XML, '<i>'
+ REPLACE(#List, #Delimiter, '</i><i>')
+ '</i>').query('.')
) AS a CROSS APPLY x.nodes('i') AS y(i)
);
GO
ALTER PROCEDURE [dbo].[spFindDuplicates]
#tableName nvarchar(255),
#columnlist nvarchar(max)
AS
BEGIN
DECLARE #query AS nvarchar(MAX);
SET #columnlist = (SELECT STUFF((SELECT ','+'['+[name]+']'
FROM SYS.columns
WHERE object_id = object_id(#tableName)
AND [Name] IN
(
SELECT Item
FROM dbo.fn_SplitString(#columnlist,',')
)
FOR XML PATH('')
)
,1,1,''))
PRINT #columnlist
SET #query = 'SELECT * FROM (SELECT '+CAST(#columnlist AS NVARCHAR(MAX))+'
FROM '+CAST(#tableName AS nvarchar(MAX))+'
GROUP BY '+CAST(#columnlist AS NVARCHAR(MAX))+'
HAVING COUNT(*) > 1)Res1
ORDER BY '+#columnlist
EXEC SP_EXECUTESQL #query;
END
GO
I'm trying to create a stored procedure that compares 2 tables and makes them identical. I tried this code:
CREATE PROCEDURE SESUS.Compare2Tables
(#Table1 AS NVARCHAR(255),
#Table2 AS NVARCHAR(255),
#key AS NVARCHAR(MAX))
AS
BEGIN
IF OBJECT_ID ('tempdb..#tmp') IS NOT NULL
DROP TABLE #tmp
--print 'SELECT * INTO #tmp FROM ' + #Table2 + ' except select * from ' + #Table1 + ';'
--print 'delete from ' + #table1 + ' where ' + #key + ' in (select ' + #key + ' from #tmp);'
--print 'insert into ' +#table1 + ' select * from ' +#table2 + ' where ' +#key + ' in (select ' +#key + ' from #tmp);'
exec ('SELECT * INTO #tmp FROM ' + #Table2 + ' except select * from ' + #Table1 + ';')
exec ('delete from ' + #table1 + ' where ' + #key + ' in (select ' + #key + ' from #tmp)')
exec ('insert into ' +#table1 + ' select * from ' +#table2 + ' where ' +#key + ' in (select ' +#key + ' from #tmp)')
END
But it returns the following error, any idea why it could insert into this temporary table, but cannot select form it?
Invalid object name '#tmp'.
You can use MERGE statement to sync with the second table.
MERGE table1 AS target
USING table2 AS source
ON source.id = target.id
WHEN MATCHED THEN
UPDATE SET col1 = source.col1, col2 = ...
WHEN NOT MATCHED BY TARGET THEN
INSERT (id, col1, ...) VALUES (source.id, col1, ...)
WHEN NOT MATCHED BY SOURCE THEN
DELETE
;
try to execute all statements inside one execution because of the limited scope of temp tables:
exec ('SELECT * INTO #tmp FROM ' + #Table2 + ' except select * from ' + #Table1 + ';
delete from ' + #table1 + ' where ' + #key + ' in (select ' + #key + ' from #tmp)
insert into ' +#table1 + ' select * from ' +#table2 + ' where ' +#key + ' in (select ' +#key + ' from #tmp)')
You can also use global temp (##tmp) tables.
If your query returns null it fails to create temp table.
create Procedure SESUS.Compare2Tables(
#Table1 as NVarchar(255),
#Table2 as NVarchar(255),
#key as NVarchar(max)
)
AS
BEGIN
IF OBJECT_ID ('tempdb..#tmp') is not null
DROP TABLE #tmp
--print 'SELECT * INTO #tmp FROM ' + #Table2 + ' except select * from ' + #Table1 + ';'
--print 'delete from ' + #table1 + ' where ' + #key + ' in (select ' + #key + ' from #tmp);'
--print 'insert into ' +#table1 + ' select * from ' +#table2 + ' where ' +#key + ' in (select ' +#key + ' from #tmp);'
exec ('SELECT * INTO #tmp FROM ' + #Table2 + ' except select * from ' + #Table1 + ';')
If(OBJECT_ID('tempdb..#tmp') Is Not Null)
Begin
exec ('delete from ' + #table1 + ' where ' + #key + ' in (select ' + #key + ' from #tmp)')
exec ('insert into ' +#table1 + ' select * from ' +#table2 + ' where ' +#key + ' in (select ' +#key + ' from #tmp)')
End
END
I'm looking for an efficient way to convert rows to columns in SQL server, I heard that PIVOT is not very fast, and I need to deal with lot of records.
This is my example:
-------------------------------
| Id | Value | ColumnName |
-------------------------------
| 1 | John | FirstName |
| 2 | 2.4 | Amount |
| 3 | ZH1E4A | PostalCode |
| 4 | Fork | LastName |
| 5 | 857685 | AccountNumber |
-------------------------------
This is my result:
---------------------------------------------------------------------
| FirstName |Amount| PostalCode | LastName | AccountNumber |
---------------------------------------------------------------------
| John | 2.4 | ZH1E4A | Fork | 857685 |
---------------------------------------------------------------------
How can I build the result?
There are several ways that you can transform data from multiple rows into columns.
Using PIVOT
In SQL Server you can use the PIVOT function to transform the data from rows to columns:
select Firstname, Amount, PostalCode, LastName, AccountNumber
from
(
select value, columnname
from yourtable
) d
pivot
(
max(value)
for columnname in (Firstname, Amount, PostalCode, LastName, AccountNumber)
) piv;
See Demo.
Pivot with unknown number of columnnames
If you have an unknown number of columnnames that you want to transpose, then you can use dynamic SQL:
DECLARE #cols AS NVARCHAR(MAX),
#query AS NVARCHAR(MAX)
select #cols = STUFF((SELECT ',' + QUOTENAME(ColumnName)
from yourtable
group by ColumnName, id
order by id
FOR XML PATH(''), TYPE
).value('.', 'NVARCHAR(MAX)')
,1,1,'')
set #query = N'SELECT ' + #cols + N' from
(
select value, ColumnName
from yourtable
) x
pivot
(
max(value)
for ColumnName in (' + #cols + N')
) p '
exec sp_executesql #query;
See Demo.
Using an aggregate function
If you do not want to use the PIVOT function, then you can use an aggregate function with a CASE expression:
select
max(case when columnname = 'FirstName' then value end) Firstname,
max(case when columnname = 'Amount' then value end) Amount,
max(case when columnname = 'PostalCode' then value end) PostalCode,
max(case when columnname = 'LastName' then value end) LastName,
max(case when columnname = 'AccountNumber' then value end) AccountNumber
from yourtable
See Demo.
Using multiple joins
This could also be completed using multiple joins, but you will need some column to associate each of the rows which you do not have in your sample data. But the basic syntax would be:
select fn.value as FirstName,
a.value as Amount,
pc.value as PostalCode,
ln.value as LastName,
an.value as AccountNumber
from yourtable fn
left join yourtable a
on fn.somecol = a.somecol
and a.columnname = 'Amount'
left join yourtable pc
on fn.somecol = pc.somecol
and pc.columnname = 'PostalCode'
left join yourtable ln
on fn.somecol = ln.somecol
and ln.columnname = 'LastName'
left join yourtable an
on fn.somecol = an.somecol
and an.columnname = 'AccountNumber'
where fn.columnname = 'Firstname'
This is rather a method than just a single script but gives you much more flexibility.
First of all There are 3 objects:
User defined TABLE type [ColumnActionList] -> holds data as
parameter
SP [proc_PivotPrepare] -> prepares our data
SP [proc_PivotExecute] -> execute the script
CREATE TYPE [dbo].[ColumnActionList] AS TABLE
(
[ID] [smallint] NOT NULL,
[ColumnName] nvarchar NOT NULL,
[Action] nchar NOT NULL
);
GO
CREATE PROCEDURE [dbo].[proc_PivotPrepare]
(
#DB_Name nvarchar(128),
#TableName nvarchar(128)
)
AS
SELECT #DB_Name = ISNULL(#DB_Name,db_name())
DECLARE #SQL_Code nvarchar(max)
DECLARE #MyTab TABLE (ID smallint identity(1,1), [Column_Name] nvarchar(128), [Type] nchar(1), [Set Action SQL] nvarchar(max));
SELECT #SQL_Code = 'SELECT [<| SQL_Code |>] = '' '' '
+ 'UNION ALL '
+ 'SELECT ''----------------------------------------------------------------------------------------------------'' '
+ 'UNION ALL '
+ 'SELECT ''-----| Declare user defined type [ID] / [ColumnName] / [PivotAction] '' '
+ 'UNION ALL '
+ 'SELECT ''----------------------------------------------------------------------------------------------------'' '
+ 'UNION ALL '
+ 'SELECT ''DECLARE #ColumnListWithActions ColumnActionList;'''
+ 'UNION ALL '
+ 'SELECT ''----------------------------------------------------------------------------------------------------'' '
+ 'UNION ALL '
+ 'SELECT ''-----| Set [PivotAction] (''''S'''' as default) to select dimentions and values '' '
+ 'UNION ALL '
+ 'SELECT ''-----|'''
+ 'UNION ALL '
+ 'SELECT ''-----| ''''S'''' = Stable column || ''''D'''' = Dimention column || ''''V'''' = Value column '' '
+ 'UNION ALL '
+ 'SELECT ''----------------------------------------------------------------------------------------------------'' '
+ 'UNION ALL '
+ 'SELECT ''INSERT INTO #ColumnListWithActions VALUES ('' + CAST( ROW_NUMBER() OVER (ORDER BY [NAME]) as nvarchar(10)) + '', '' + '''''''' + [NAME] + ''''''''+ '', ''''S'''');'''
+ 'FROM [' + #DB_Name + '].sys.columns '
+ 'WHERE object_id = object_id(''[' + #DB_Name + ']..[' + #TableName + ']'') '
+ 'UNION ALL '
+ 'SELECT ''----------------------------------------------------------------------------------------------------'' '
+ 'UNION ALL '
+ 'SELECT ''-----| Execute sp_PivotExecute with parameters: columns and dimentions and main table name'' '
+ 'UNION ALL '
+ 'SELECT ''----------------------------------------------------------------------------------------------------'' '
+ 'UNION ALL '
+ 'SELECT ''EXEC [dbo].[sp_PivotExecute] #ColumnListWithActions, ' + '''''' + #TableName + '''''' + ';'''
+ 'UNION ALL '
+ 'SELECT ''----------------------------------------------------------------------------------------------------'' '
EXECUTE SP_EXECUTESQL #SQL_Code;
GO
CREATE PROCEDURE [dbo].[sp_PivotExecute]
(
#ColumnListWithActions ColumnActionList ReadOnly
,#TableName nvarchar(128)
)
AS
--#######################################################################################################################
--###| Step 1 - Select our user-defined-table-variable into temp table
--#######################################################################################################################
IF OBJECT_ID('tempdb.dbo.#ColumnListWithActions', 'U') IS NOT NULL DROP TABLE #ColumnListWithActions;
SELECT * INTO #ColumnListWithActions FROM #ColumnListWithActions;
--#######################################################################################################################
--###| Step 2 - Preparing lists of column groups as strings:
--#######################################################################################################################
DECLARE #ColumnName nvarchar(128)
DECLARE #Destiny nchar(1)
DECLARE #ListOfColumns_Stable nvarchar(max)
DECLARE #ListOfColumns_Dimension nvarchar(max)
DECLARE #ListOfColumns_Variable nvarchar(max)
--############################
--###| Cursor for List of Stable Columns
--############################
DECLARE ColumnListStringCreator_S CURSOR FOR
SELECT [ColumnName]
FROM #ColumnListWithActions
WHERE [Action] = 'S'
OPEN ColumnListStringCreator_S;
FETCH NEXT FROM ColumnListStringCreator_S
INTO #ColumnName
WHILE ##FETCH_STATUS = 0
BEGIN
SELECT #ListOfColumns_Stable = ISNULL(#ListOfColumns_Stable, '') + ' [' + #ColumnName + '] ,';
FETCH NEXT FROM ColumnListStringCreator_S INTO #ColumnName
END
CLOSE ColumnListStringCreator_S;
DEALLOCATE ColumnListStringCreator_S;
--############################
--###| Cursor for List of Dimension Columns
--############################
DECLARE ColumnListStringCreator_D CURSOR FOR
SELECT [ColumnName]
FROM #ColumnListWithActions
WHERE [Action] = 'D'
OPEN ColumnListStringCreator_D;
FETCH NEXT FROM ColumnListStringCreator_D
INTO #ColumnName
WHILE ##FETCH_STATUS = 0
BEGIN
SELECT #ListOfColumns_Dimension = ISNULL(#ListOfColumns_Dimension, '') + ' [' + #ColumnName + '] ,';
FETCH NEXT FROM ColumnListStringCreator_D INTO #ColumnName
END
CLOSE ColumnListStringCreator_D;
DEALLOCATE ColumnListStringCreator_D;
--############################
--###| Cursor for List of Variable Columns
--############################
DECLARE ColumnListStringCreator_V CURSOR FOR
SELECT [ColumnName]
FROM #ColumnListWithActions
WHERE [Action] = 'V'
OPEN ColumnListStringCreator_V;
FETCH NEXT FROM ColumnListStringCreator_V
INTO #ColumnName
WHILE ##FETCH_STATUS = 0
BEGIN
SELECT #ListOfColumns_Variable = ISNULL(#ListOfColumns_Variable, '') + ' [' + #ColumnName + '] ,';
FETCH NEXT FROM ColumnListStringCreator_V INTO #ColumnName
END
CLOSE ColumnListStringCreator_V;
DEALLOCATE ColumnListStringCreator_V;
SELECT #ListOfColumns_Variable = LEFT(#ListOfColumns_Variable, LEN(#ListOfColumns_Variable) - 1);
SELECT #ListOfColumns_Dimension = LEFT(#ListOfColumns_Dimension, LEN(#ListOfColumns_Dimension) - 1);
SELECT #ListOfColumns_Stable = LEFT(#ListOfColumns_Stable, LEN(#ListOfColumns_Stable) - 1);
--#######################################################################################################################
--###| Step 3 - Preparing table with all possible connections between Dimension columns excluding NULLs
--#######################################################################################################################
DECLARE #DIM_TAB TABLE ([DIM_ID] smallint, [ColumnName] nvarchar(128))
INSERT INTO #DIM_TAB
SELECT [DIM_ID] = ROW_NUMBER() OVER(ORDER BY [ColumnName]), [ColumnName] FROM #ColumnListWithActions WHERE [Action] = 'D';
DECLARE #DIM_ID smallint;
SELECT #DIM_ID = 1;
DECLARE #SQL_Dimentions nvarchar(max);
IF OBJECT_ID('tempdb.dbo.##ALL_Dimentions', 'U') IS NOT NULL DROP TABLE ##ALL_Dimentions;
SELECT #SQL_Dimentions = 'SELECT [xxx_ID_xxx] = ROW_NUMBER() OVER (ORDER BY ' + #ListOfColumns_Dimension + '), ' + #ListOfColumns_Dimension
+ ' INTO ##ALL_Dimentions '
+ ' FROM (SELECT DISTINCT' + #ListOfColumns_Dimension + ' FROM ' + #TableName
+ ' WHERE ' + (SELECT [ColumnName] FROM #DIM_TAB WHERE [DIM_ID] = #DIM_ID) + ' IS NOT NULL ';
SELECT #DIM_ID = #DIM_ID + 1;
WHILE #DIM_ID <= (SELECT MAX([DIM_ID]) FROM #DIM_TAB)
BEGIN
SELECT #SQL_Dimentions = #SQL_Dimentions + 'AND ' + (SELECT [ColumnName] FROM #DIM_TAB WHERE [DIM_ID] = #DIM_ID) + ' IS NOT NULL ';
SELECT #DIM_ID = #DIM_ID + 1;
END
SELECT #SQL_Dimentions = #SQL_Dimentions + ' )x';
EXECUTE SP_EXECUTESQL #SQL_Dimentions;
--#######################################################################################################################
--###| Step 4 - Preparing table with all possible connections between Stable columns excluding NULLs
--#######################################################################################################################
DECLARE #StabPos_TAB TABLE ([StabPos_ID] smallint, [ColumnName] nvarchar(128))
INSERT INTO #StabPos_TAB
SELECT [StabPos_ID] = ROW_NUMBER() OVER(ORDER BY [ColumnName]), [ColumnName] FROM #ColumnListWithActions WHERE [Action] = 'S';
DECLARE #StabPos_ID smallint;
SELECT #StabPos_ID = 1;
DECLARE #SQL_MainStableColumnTable nvarchar(max);
IF OBJECT_ID('tempdb.dbo.##ALL_StableColumns', 'U') IS NOT NULL DROP TABLE ##ALL_StableColumns;
SELECT #SQL_MainStableColumnTable = 'SELECT xxx_ID_xxx = ROW_NUMBER() OVER (ORDER BY ' + #ListOfColumns_Stable + '), ' + #ListOfColumns_Stable
+ ' INTO ##ALL_StableColumns '
+ ' FROM (SELECT DISTINCT' + #ListOfColumns_Stable + ' FROM ' + #TableName
+ ' WHERE ' + (SELECT [ColumnName] FROM #StabPos_TAB WHERE [StabPos_ID] = #StabPos_ID) + ' IS NOT NULL ';
SELECT #StabPos_ID = #StabPos_ID + 1;
WHILE #StabPos_ID <= (SELECT MAX([StabPos_ID]) FROM #StabPos_TAB)
BEGIN
SELECT #SQL_MainStableColumnTable = #SQL_MainStableColumnTable + 'AND ' + (SELECT [ColumnName] FROM #StabPos_TAB WHERE [StabPos_ID] = #StabPos_ID) + ' IS NOT NULL ';
SELECT #StabPos_ID = #StabPos_ID + 1;
END
SELECT #SQL_MainStableColumnTable = #SQL_MainStableColumnTable + ' )x';
EXECUTE SP_EXECUTESQL #SQL_MainStableColumnTable;
--#######################################################################################################################
--###| Step 5 - Preparing table with all options ID
--#######################################################################################################################
DECLARE #FULL_SQL_1 NVARCHAR(MAX)
SELECT #FULL_SQL_1 = ''
DECLARE #i smallint
IF OBJECT_ID('tempdb.dbo.##FinalTab', 'U') IS NOT NULL DROP TABLE ##FinalTab;
SELECT #FULL_SQL_1 = 'SELECT t.*, dim.[xxx_ID_xxx] '
+ ' INTO ##FinalTab '
+ 'FROM ' + #TableName + ' t '
+ 'JOIN ##ALL_Dimentions dim '
+ 'ON t.' + (SELECT [ColumnName] FROM #DIM_TAB WHERE [DIM_ID] = 1) + ' = dim.' + (SELECT [ColumnName] FROM #DIM_TAB WHERE [DIM_ID] = 1);
SELECT #i = 2
WHILE #i <= (SELECT MAX([DIM_ID]) FROM #DIM_TAB)
BEGIN
SELECT #FULL_SQL_1 = #FULL_SQL_1 + ' AND t.' + (SELECT [ColumnName] FROM #DIM_TAB WHERE [DIM_ID] = #i) + ' = dim.' + (SELECT [ColumnName] FROM #DIM_TAB WHERE [DIM_ID] = #i)
SELECT #i = #i +1
END
EXECUTE SP_EXECUTESQL #FULL_SQL_1
--#######################################################################################################################
--###| Step 6 - Selecting final data
--#######################################################################################################################
DECLARE #STAB_TAB TABLE ([STAB_ID] smallint, [ColumnName] nvarchar(128))
INSERT INTO #STAB_TAB
SELECT [STAB_ID] = ROW_NUMBER() OVER(ORDER BY [ColumnName]), [ColumnName]
FROM #ColumnListWithActions WHERE [Action] = 'S';
DECLARE #VAR_TAB TABLE ([VAR_ID] smallint, [ColumnName] nvarchar(128))
INSERT INTO #VAR_TAB
SELECT [VAR_ID] = ROW_NUMBER() OVER(ORDER BY [ColumnName]), [ColumnName]
FROM #ColumnListWithActions WHERE [Action] = 'V';
DECLARE #y smallint;
DECLARE #x smallint;
DECLARE #z smallint;
DECLARE #FinalCode nvarchar(max)
SELECT #FinalCode = ' SELECT ID1.*'
SELECT #y = 1
WHILE #y <= (SELECT MAX([xxx_ID_xxx]) FROM ##FinalTab)
BEGIN
SELECT #z = 1
WHILE #z <= (SELECT MAX([VAR_ID]) FROM #VAR_TAB)
BEGIN
SELECT #FinalCode = #FinalCode + ', [ID' + CAST((#y) as varchar(10)) + '.' + (SELECT [ColumnName] FROM #VAR_TAB WHERE [VAR_ID] = #z) + '] = ID' + CAST((#y + 1) as varchar(10)) + '.' + (SELECT [ColumnName] FROM #VAR_TAB WHERE [VAR_ID] = #z)
SELECT #z = #z + 1
END
SELECT #y = #y + 1
END
SELECT #FinalCode = #FinalCode +
' FROM ( SELECT * FROM ##ALL_StableColumns)ID1';
SELECT #y = 1
WHILE #y <= (SELECT MAX([xxx_ID_xxx]) FROM ##FinalTab)
BEGIN
SELECT #x = 1
SELECT #FinalCode = #FinalCode
+ ' LEFT JOIN (SELECT ' + #ListOfColumns_Stable + ' , ' + #ListOfColumns_Variable
+ ' FROM ##FinalTab WHERE [xxx_ID_xxx] = '
+ CAST(#y as varchar(10)) + ' )ID' + CAST((#y + 1) as varchar(10))
+ ' ON 1 = 1'
WHILE #x <= (SELECT MAX([STAB_ID]) FROM #STAB_TAB)
BEGIN
SELECT #FinalCode = #FinalCode + ' AND ID1.' + (SELECT [ColumnName] FROM #STAB_TAB WHERE [STAB_ID] = #x) + ' = ID' + CAST((#y+1) as varchar(10)) + '.' + (SELECT [ColumnName] FROM #STAB_TAB WHERE [STAB_ID] = #x)
SELECT #x = #x +1
END
SELECT #y = #y + 1
END
SELECT * FROM ##ALL_Dimentions;
EXECUTE SP_EXECUTESQL #FinalCode;
From executing the first query (by passing source DB and table name) you will get a pre-created execution query for the second SP, all you have to do is define is the column from your source:
+ Stable
+ Value (will be used to concentrate values based on that)
+ Dim (column you want to use to pivot by)
Names and datatypes will be defined automatically!
I cant recommend it for any production environments but does the job for adhoc BI requests.
I modified Taryn's answer ("Pivot with unknown number of columnnames" version) to show more than 1 row in the result. This requires to have an additional "Group" column
DROP TABLE #yourtable
CREATE table #yourtable
([Id] int,[Group] int, [Value] varchar(6), [ColumnName] varchar(13))
;
INSERT INTO #yourtable
([Id],[Group], [Value], [ColumnName])
VALUES
(1,1, 'John', 'FirstName'),
(2,1, '2.4', 'Amount'),
(3,1, 'ZH1E4A', 'PostalCode'),
(4,1, 'Fork', 'LastName'),
(5,1, '857685', 'AccountNumber'),
(6,2, 'Pedro', 'FirstName'),
(7,2, '5.1', 'Amount'),
(8,2, '123456', 'PostalCode'),
(9,2, 'Torres', 'LastName'),
(10,2, '857686', 'AccountNumber')
;
;
DECLARE #cols AS NVARCHAR(MAX),
#query AS NVARCHAR(MAX)
select #cols = STUFF((SELECT ',' + QUOTENAME(ColumnName)
from #yourtable
group by [Group], ColumnName, id
having [group] = (SELECT TOP 1 MIN([Group])FROM #yourtable)
order by id
FOR XML PATH(''), TYPE
).value('.', 'NVARCHAR(MAX)')
,1,1,'')
set #query = N'SELECT ' + #cols + N' from
(
select value, ColumnName,[Group]
from #yourtable
GROUP BY [Group],ColumnName,Value
) x
pivot
(
max(value)
for ColumnName in (' + #cols + N')
) p '
exec sp_executesql #query;
Please try
CREATE TABLE pvt (Present int, [Absent] int);
GO
INSERT INTO pvt VALUES (10,40);
GO
--Unpivot the table.
SELECT Code, Value
FROM
(SELECT Present, Absent
FROM pvt) p
UNPIVOT
(Value FOR Code IN
(Present, [Absent])
)AS unpvt;
GO
DROP TABLE pvt
One more option which could be very useful is using CROSS APPLY
-- Original data
SELECT * FROM (VALUES ('1', 1, 2, 3),('2', 11, 22, 33)) AS Stage(id,col1,col2,col3)
-- row to columns using CROSS APPLY
SELECT Stage.id,v.idd, v.colc
FROM (VALUES ('1', 1, 2, 3),('2', 11, 22, 33)) AS Stage(id,col1,col2,col3)
CROSS APPLY (VALUES ('col1', col1),('col2', col2),('col3', col3)) AS v(idd,colc)
GO
I have 20 databases, each with same table but different columns.
So to make the uniform we are creating views on top of each table in a database which will contain all the columns, as there will be one application accessing all the database.
In the view, I have to write the query in such a way that if I want to alter it and add any addition column for testing I should be able to do that.
Now in below query I am altering / creating query such that it takes all the columns of that table from the database, and then I append the other columns which are not present in it.
I need to add a column which will just concatenate some of the columns
ALTER VIEW [dbo].[AIV_PARKING]
AS
SELECT
*,
Cast(NULL AS [VARCHAR](20)) [ACTCODE],
Cast(NULL AS [VARCHAR](1)) [ACTIVATEINFO],
Cast(NULL AS [VARCHAR](20)) [VEHLICNOCHECK],
Cast(NULL AS [VARCHAR](40)) [ACTIVITY],
Cast(Isnull(vehlicnocheck, '') + '|' +
Isnull(officername, '') + '|' +
Isnull(locstreet, '') + '|' +
Isnull(locsideofstreet, '') + '|' +
Isnull(loccrossstreet1, '') + '|' +
Isnull(loccrossstreet2, '') + '|'
+ Isnull(locsuburb, '') + '|'
+ Isnull(locstate, '') + '|'
+ Isnull(locpostalcode, '') + '|'
+ Isnull(loclot, '') + '|'
+ Isnull(locpostalcode, '') + '|'
+ Isnull(Cast(officerid AS VARCHAR(20)), '')
+ Isnull(officername, '') + '|'
+ Isnull(Cast (issueno AS VARCHAR(100)), '') AS NVARCHAR(max)) AS SearchText
FROM
[dbo].parking
Here I added a column called SearchText which concatenates other columns, but I get an error
Invalid column name 'VehLicNoCheck'
Is there any way I can add this column to this view?
I also tried to do to something below but I got the same error:
CAST(CASE
WHEN NOT EXISTS
(
Select 1 from INFORMATION_SCHEMA.COLUMNS
Where Column_name ='VehLicNoCheck'
and table_name='Parking'
)
THEN ''
ELSE ISNULL(VehLicNoCheck,'')
END as nvarchar(max)
)
you could create a view that normalizes the uncommon columns to rows, where the values for the common columns are just repeated, e.g:
select id, col, value from parking
unpivot (value for col in (actcode, vehLicNoCheck, etc.)) x
the code to dynamically generate the view would be something like:
declare #sql varchar(max) = 'select id, col, value from parking unpivot (value for col in ('
select #sql += quotename(name) +',' from sys.columns where object_id=object_id('parking') and name not in ('id')
set #sql = substring(#sql, 1, len(#sql) - 1) + '))x'
exec(#sql)
this does not make sense at all.
the [ACTCODE], [ACTIVATEINFO] are all NULL value
so basically SearchText is just a string of '|||||'
you might as well, just do this
SELECT *,
CAST( NULL AS varchar) [ACTCODE],
CAST( NULL AS varchar) [ACTIVATEINFO],
CAST( NULL AS varchar) [VEHLICNOCHECK],
CAST( NULL AS varchar) [ACTIVITY],
'||||||' as SearchText
FROM [dbo].PARKING
Maybe if you can explain what are you trying to achieve here, we can point you to the right direction
EDIT :
You will need to use Dynamic SQL. You will need a list of all column names
-- declare a table variable for all the columns that you required
declare #columns table
(
id int identity,
name varchar(100)
)
-- for example these are the required columns
insert into #columns
values ('ACTCODE'), ('ACTIVATEINFO'), ('VEHLICNOCHECK'), ('ACTIVITY')
-- The Query to create the view
declare #sql nvarchar(max)
select #sql = N'CREATE VIEW [AIV_PARKING] AS' + char(13) + 'SELECT' + char(13)
select #sql = #sql
+ case when t.name is not null
then quotename(c.name) + ','
else 'CAST (NULL AS VARCHAR(10)) AS ' + quotename(c.name) + ','
end
+ char(13)
from #columns c
left join sys.columns t on c.name = t.name
and t.object_id = object_id('PARKING')
order by c.id
select #sql = #sql
+ case when t.name is not null
then 'ISNULL(' + quotename(c.name) + ', '''')'
else ''
end
+ ' + ''|'''
+ ' + '
from #columns c
left join sys.columns t on c.name = t.name
and t.object_id = object_id('PARKING')
order by c.id
select #sql = left(#sql, len(#sql) - 8) + ' AS SearchText' + char(13)
+ 'FROM PARKING'
-- print out to view the complete create view statement
print #sql
-- execute it
exec sp_executesql #sql