Insert/update based on dynamic XML - sql-server

I have following XML:
<NewDataSet>
<Data>
<Id>560f05b2-b215-4fea-9ac6-7f012fbca331</Id>
<Number>384D25334E04593B6DE9955E72F413F8A0A828FF</Number>
<CurrentDate>2012-11-21T09:09:26+00:00</CurrentDate>
</Data>
<Data>
<Id>9cff574b-59ea-4cbd-a2db-9ed02b6cc602</Id>
<Number>384D25334E04593B6DE9955E72F413F8A0A828FF</Number>
<Location>Town</Location>
<CurrentDate>2012-11-21T09:09:53+00:00</CurrentDate>
</Data>
</NewDataSet>
I'm trying to write a query that will inster new record or update existing one based on given xml. The problem is I cannot use predefined names of columns, becuase table straucture sometimes is changing. So the idea is to generate dynamic query and apply it. So far I've got following thing:
SET NOCOUNT OFF;
DECLARE #TableName nvarchar(50)
DECLARE #TableData xml
DECLARE #Query nvarchar(max)
DECLARE #Id uniqueidentifier
DECLARE #CurrentDate datetime
-- declare cursor
DECLARE cursor_inserting CURSOR LOCAL FAST_FORWARD FOR
SELECT
r.value('fn:local-name(.)', 'nvarchar(50)'),
r.query('.')
FROM #Data.nodes('//NewDataSet/*') AS records(r)
ORDER BY r.value('fn:local-name(.)', 'nvarchar(50)')
-- open cursor
OPEN cursor_inserting
FETCH NEXT FROM cursor_inserting INTO #TableName, #TableData
WHILE ##FETCH_STATUS = 0
BEGIN
-- Get id
SELECT #Id = o.value('Id[1]', 'uniqueidentifier') FROM #TableData.nodes('*') as n(o)
SELECT #CurrentDate = o.value('CurrentDate[1]', 'datetime') FROM #TableData.nodes('*') as n(o)
SET #Query = NULL
-- temporary update query
SET #UpdateTemp = NULL
SELECT #UpdateTemp = COALESCE(#UpdateTemp + ', ', '') + o.value('fn:local-name(.)', 'nvarchar(50)') + ' = ''' + CAST(o.query('text()') as nvarchar(4000)) + '''' FROM #TableData.nodes('/*/*') as n(o)
SET #UpdateTemp = 'UPDATE ' + #TableName + ' SET ' + #UpdateTemp + ' WHERE Id = ''' + CAST(#Id as nvarchar(40)) + ''''
-- temporary insert query
SET #Insert1Temp = NULL
SELECT #Insert1Temp = COALESCE(#Insert1Temp + ', ', '') + o.value('fn:local-name(.)', 'nvarchar(50)') FROM #TableData.nodes('/*/*') as n(o)
SET #Insert2Temp = NULL
SELECT #Insert2Temp = COALESCE(#Insert2Temp + ', ', '') + '''' + CAST(o.query('text()') as nvarchar(4000)) + '''' FROM #TableData.nodes('/*/*') as n(o)
SET #InsertTemp = 'INSERT INTO ' + #TableName + ' ( ' + #Insert1Temp + ' ) VALUES ( ' + #Insert2Temp + ' )'
IF #TableName = 'Data'
BEGIN
IF EXISTS (SELECT * FROM Data WHERE Id = #Id)
BEGIN
IF EXISTS (SELECT * FROM tblAudit WHERE Id = #Id AND CurrentDate < #CurrentDate)
BEGIN
SET #Query = #UpdateTemp
END
END
ELSE
BEGIN
SET #Query = #InsertTemp
END
END
IF #Query IS NOT NULL
BEGIN
SELECT #Query
EXEC (#Query)
END
END
FETCH NEXT FROM cursor_inserting INTO #TableName, #TableData
END
CLOSE cursor_inserting
DEALLOCATE cursor_inserting
If there is any better way to achive this inside SQL I would like to know, I know that I can do this outside SQL in my application code, but I would like to have it in one place in stored procedure to provide xml and have required action taken.
UPDATE 1
I would like to clarify that my main problem is query proper generation based on XML. The different way of handling instert/update is nice to see, but as addition
UPDATE 2
There can be more than 1 table in xml. E.g. not only Data but also Data2
UPDATE 3
I've update what I have now - and it is now generating proper Insert/Update however I now have issues with conversion. E.g. Date string is in xml format and sql doesn't want to convert it automaticaly. So my next step is get proper column type from database and instead of generate query instert directly from xml.I hope this will work.

Yes.
You can use MERGE and SQL XQuery to do it in one statement.
Something like...
merge Data as target
using
(
select
x.q.value('Id[1]','uniqueidentifier') as ID,
x.q.value('Number[1]','varchar(50)') as Number,
x.q.value('Location[1]','varchar(50)') as Town,
x.q.value('CurrentDate[1]','datetime') as CurrentDate
from
#TableData.nodes('/NewDataSet/Data')x(q)
) as Source (ID,Number,Town,CurrentDate)
on target.id=source.id
when matched and target.CurrentDate < source.CurrentDate then
update set
Number = source.number,
town = source.town,
currentdate = source.currentdate
when not matched then
insert (ID,number,town,currentdate)
values (source.id,source.number,source.town,source.currentdate);

Related

Dynamic Database Stored Procedure on SQL Server 2016

I'm trying to build a stored procedure that will query multiple database depending on the databases required.
For example:
SP_Users takes a list of #DATABASES as parameters.
For each database it needs to run the same query and union the results together.
I believe a CTE could be my best bet so I have something like this at the moment.
SET #DATABASES = 'DB_1, DB_2' -- Two databases in a string listed
-- I have a split string function that will extract each database
SET #CURRENT_DB = 'DB_1'
WITH UsersCTE (Name, Email)
AS (SELECT Name, Email
FROM [#CURRENT_DB].[dbo].Users),
SELECT #DATABASE as DB, Name, Email
FROM UsersCTE
What I don't want to do is hard code the databases in the query. The steps I image are:
Split the parameter #DATABASES to extract and set the #CURRENT_DB Variable
Iterate through the query with a Recursive CTE until all the #DATABASES have been processed
Union all results together and return the data.
Not sure if this is the right approach to tackling this problem.
Using #databases:
As mentioned in the comments to your question, variables cant be used to dynamically select a database. Dynamic sql is indicated. You can start by building your template sql statement:
declare #sql nvarchar(max) =
'union all ' +
'select ''#db'' as db, name, email ' +
'from [#db].dbo.users ';
Since you have sql server 2016, you can split using the string_split function, with your #databases variable as input. This will result in a table with 'value' as the column name, which holds the database names.
Use the replace function to replace #db in the template with value. This will result in one sql statement for each database you passed into #databases. Then, concatenate the statements back together. Unfortunately, in version 2016, there's no built in function to do that. So we have to use the famous for xml trick to join the statements, then we use .value to convert it to a string, and finally we use stuff to get rid of the leading union all statement.
Take the results of the concatenated output, and overwrite the #sql variable. It is ready to go at this point, so execute it.
I do all that is described in this code:
declare #databases nvarchar(max) = 'db_1,db_2';
set #sql = stuff(
(
select replace(#sql, '#db', value)
from string_split(#databases, ',')
for xml path(''), type
).value('.[1]', 'nvarchar(max)')
, 1, 9, '');
exec(#sql);
Untested, of course, but if you print instead of execute, it seems to give the proper sql statement for your needs.
Using msForEachDB:
Now, if you didn't want to have to know which databases had 'users', such as if you're in an environment where you have a different database for every client, you can use sp_msForEachDb and check the structure first to make sure it has a 'users' table with 'name' and 'email' columns. If so, execute the appropriate statement. If not, execute a dummy statement. I won't describe this one, I'll just give the code:
declare #aggregator table (
db sysname,
name int,
email nvarchar(255)
);
insert #aggregator
exec sp_msforeachdb '
declare #sql nvarchar(max) = ''select db = '''''''', name = '''''''', email = '''''''' where 1 = 2'';
select #sql = ''select db = ''''?'''', name, email from ['' + table_catalog + ''].dbo.users''
from [?].information_schema.columns
where table_schema = ''dbo''
and table_name = ''users''
and column_name in (''name'', ''email'')
group by table_catalog
having count(*) = 2
exec (#sql);
';
select *
from #aggregator
I took the valid advice from others here and went with this which works great for what I need:
I decided to use a loop to build the query up. Hope this helps someone else looking to do something similar.
CREATE PROCEDURE [dbo].[SP_Users](
#DATABASES VARCHAR(MAX) = NULL,
#PARAM1 VARCHAR(250),
#PARAM2 VARCHAR(250)
)
BEGIN
SET NOCOUNT ON;
--Local variables
DECLARE
#COUNTER INT = 0,
#SQL NVARCHAR(MAX) = '',
#CURRENTDB VARCHAR(50) = NULL,
#MAX INT = 0,
#ERRORMSG VARCHAR(MAX)
--Check we have databases entered
IF #DATABASES IS NULL
BEGIN
RAISERROR('ERROR: No Databases Provided,
Please Provide a list of databases to execute procedure. See stored procedure:
[SP_Users]', 16, 1)
RETURN
END
-- SET Number of iterations based on number of returned databases
SET #MAX = (SELECT COUNT(*) FROM
(SELECT ROW_NUMBER() OVER (ORDER BY i.value) AS RowNumber, i.value
FROM dbo.udf_SplitVariable(#DATABASES, ',') AS i)X)
-- Build SQL Statement
WHILE #COUNTER < #MAX
BEGIN
--Set the current database
SET #CURRENTDB = (SELECT X.Value FROM
(SELECT ROW_NUMBER() OVER (ORDER BY i.value) AS RowNumber, i.value
FROM dbo.udf_SplitVariable(#DATABASES, ',') AS i
ORDER BY RowNumber OFFSET #COUNTER
ROWS FETCH NEXT 1 ROWS ONLY) X);
SET #SQL = #SQL + N'
(
SELECT Name, Email
FROM [' + #CURRENTDB + '].[dbo].Users
WHERE
(Name = #PARAM1 OR #PARAM1 IS NULL)
(Email = #PARAM2 OR #PARAM2 IS NULL)
) '
+ N' UNION ALL '
END
PRINT #CURRENTDB
PRINT #SQL
SET #COUNTER = #COUNTER + 1
END
-- remove last N' UNION ALL '
IF LEN(#SQL) > 11
SET #SQL = LEFT(#SQL, LEN(#SQL) - 11)
EXEC sp_executesql #SQL, N'#CURRENTDB VARCHAR(50),
#PARAM1 VARCHAR(250),
#PARAM2 VARCHAR(250)',
#CURRENTDB,
#PARAM1 ,
#PARAM2
END
Split Variable Function
CREATE FUNCTION [dbo].[udf_SplitVariable]
(
#List varchar(8000),
#SplitOn varchar(5) = ','
)
RETURNS #RtnValue TABLE
(
Id INT IDENTITY(1,1),
Value VARCHAR(8000)
)
AS
BEGIN
--Account for ticks
SET #List = (REPLACE(#List, '''', ''))
--Account for 'emptynull'
IF LTRIM(RTRIM(#List)) = 'emptynull'
BEGIN
SET #List = ''
END
--Loop through all of the items in the string and add records for each item
WHILE (CHARINDEX(#SplitOn,#List)>0)
BEGIN
INSERT INTO #RtnValue (value)
SELECT Value = LTRIM(RTRIM(SUBSTRING(#List, 1, CHARINDEX(#SplitOn, #List)-1)))
SET #List = SUBSTRING(#List, CHARINDEX(#SplitOn,#List) + LEN(#SplitOn), LEN(#List))
END
INSERT INTO #RtnValue (Value)
SELECT Value = LTRIM(RTRIM(#List))
RETURN
END

How to Pass Tablename, Fieldnames, Values as perameters in Stored Procedure using CI?

I am developing a custom application using CodeIgniter and MSSQL Server. Here i am using stored procedures.
Now i am wondering to implement codeigniter query type functionality where i can create a universal stored procedure in SQL Server and at the time of using i can pass tablename, array of fields and values.
It can work for both insert and update.
Something like we do in CodeIgniter to execute the query,
$data = array('fieldname1' => 'value1',
'fieldname2' => 'value2');
$this->db->insert($tablename,$data);
Just like this if we can pass the table name and array of the data to stored procedure and stored procedure automatically execute it.
If this can be done, it can save lots n lots of man hours. If anyone have already done i will be very much happy to see the solution.
You need to make string very specific in this case.
Figure out your table name, Column name, Column values for insert. For update 2 more parameters are required Id column name and its value.
GO
---- exec InsertUpdate 'tablename', 'col1, col2, col3', 'val1, val2, val3', 'idcol', 'idval'
GO
Create proc InsertUpdate
( #TableName nvarchar(500),
#ColName nvarchar(max),
#ColValues nvarchar(max),
#IDColName nvarchar(100) = '', --- for update only otherwise null
#IdColValue nvarchar(Max) = '' --- for update only otherwise null
)
As
Begin
declare #Query nvarchar(max)
if (#IdColValue = '')
Begin
set #Query = ' Insert into ' + #TableName + ' (' + #ColName + ') values (' + #ColValues + ')'
End
Else
Begin
;with CtColumn as (
select ROW_NUMBER() over (order by (select 1000)) as Slno, * from Split(#ColName,',') )
, CtValue as (
select ROW_NUMBER() over (order by (select 1000)) as Slno, * from Split(#ColValues, ','))
, CTFinal as (
select CCOl.Slno, CCOl.Items as ColName, CVal.Items as ColValue from CtColumn as CCOl inner join CtValue as CVal on CCOl.Slno=CVal.Slno )
select #Query = 'update ' + #TableName + ' set ' +
stuff ( (select ',' + ColName + '=' + ColValue from CTFinal for xml path ('')) ,1,1,'') +
' where ' + #IDColName + '=' + #IdColValue
End
exec sp_executesql #Query
End
Go

Querying a column that contains comma separated values

I'm trying to figure out why my where clause is returning all rows.
I'm querying a column that contains csv's using a variable that also contains csv's. I've built a stored function to split the variable on csv and return a table with one row that contains what I'd like to have on the right side of the LIKE operator.
Example:
The stored function:
ALTER Function [dbo].[storedFunction]
(#Fields VARCHAR(MAX),
#Field_Name VARCHAR(MAX) = '')
RETURN #Tbl_Fields Table (FIELD Varchar(max))
AS
BEGIN
DECLARE #FIELD varchar(max) = REPLACE(#Fields, ',', '%''' + ' AND ' +
#Field_Name + ' Like ' + '''%');
INSERT INTO #Tbl_Fields
SELECT '''%' + #FIELD + '%'''
RETURN
END
Using the stored function:
BEGIN
DECLARE #variable varchar(max) = 'variable1, variable3';
END
SELECT field
FROM storedFunction(#variable, 'main_csv_field');
returns '%variable1%' AND main_csv_field Like '%variable3%'
My simplified query:
BEGIN
DECLARE #variable varchar(max) = 'variable1, variable3';
END
SELECT main_csv_field
FROM table
WHERE (main_csv_field LIKE (SELECT field
FROM storedFunction(#variable, 'main_csv_field');
returns
variable1,variable2,variable3,variable4,...
variable2,variable4,...
variable1,variable3,...
My problem is this last query returns all of the rows in the table regardless of value matching. Were I to copy and paste the value returned from the stored function I would get the data that I need.
How/what is the difference here?
Thanks to #Obie and #AllanS.Hansen I knew where to start looking to fix this. Its pretty rough, but I wanted to post a solution before I got too far down the rabbit hole:
DECLARE variable1 varchar(max) = '' --around 9 passed from code
DECLARE #query nvarchar(max);
DECLARE #column_list varchar(max) = 'column1, column2, etc'
--one of each of the tests per variable passed from code
DECLARE #variable1_test nvarchar(max) = (SELECT CASE WHEN #variable = '' THEN '%' ELSE (SELECT * from dbo.stored_function(#variable, 'column_name')) END);
END;
SET #query = ' SELECT ' + #column_list + '
FROM table_name
WHERE variable LIKE ''' + #variable_test + ''' '
EXECUTE sp_executesql #query;
print(#query); --This is just to see the resulting query, which helped me a ton
Exciting! Now I have to test it.

Avoid hardcoding when changing values in SQL Server trigger

I have a sql server "instead of insert" trigger that populates a single column (PromoCode). It all works perfectly, but I don't like the fact I have had to hardcode the columns in actual INSERT statement:
CREATE TRIGGER PopulateOrderPromoCode ON Order
INSTEAD OF INSERT
AS BEGIN
--// Get the Promo Code
DECLARE #PromoCode int;
EXEC GetPromoCode #PromoCode OUTPUT;
--// Insert the order with the new Promo Code
INSERT INTO Order (Id, CustomerId, PromoCode)
SELECT Id, CustomerId, #PromoCode FROM inserted;
END
I would prefer to simply replace the value inside inserted.PromoCode with #PromoCode and then could use:
INSERT INTO Order
SELECT * FROM inserted;
Can this be done?
Don't use an INSTEAD OF INSERT trigger (in which you have to take over the insert logic)
Use a normal INSERT trigger (which allows you to do stuff in addition to the insert)
This assumes you can insert without a promo code (allows nulls) or the promo code defaults to something.
CREATE TRIGGER PopulateOrderPromoCode ON Order
FOR INSERT
AS
BEGIN
--// Get the Promo Code
DECLARE #PromoCode int;
EXEC GetPromoCode #PromoCode OUTPUT;
--// update the order with the new Promo Code
UPDATE Order SET PromoCode = #PromoCode
WHERE ID IN (SELECT ID FROM inserted)
END
INSERTED is a read only temporary table which can be accessed in any trigger. You can not modify it.
And the way you are using to perform an INSERT is the best way. So, there's nothing wrong with it. Its good to specify the columns while performing an INSERT (according to me).
Dynamic SQL would be your only other option. Try this:
CREATE TRIGGER PopulateOrderPromoCode
ON Order
INSTEAD OF INSERT
AS
BEGIN
--// Get the Promo Code
DECLARE #PromoCode int;
EXEC GetPromoCode #PromoCode OUTPUT;
DECLARE #InsertSQL nvarchar(2000), #SelectSQL nvarchar(2000)
SET #InsertSQL = 'INSERT INTO Order ('
SET #SelectSQL = 'SELECT '
DECLARE #CurrentCol sysname
SET #CurrentCol = ''
WHILE EXISTS ( SELECT TOP 1 QUOTENAME(name)
FROM sys.syscolumns
WHERE object_name(id) = 'Order'
AND name <> 'PromoCode'
AND name > #CurrentCol)
BEGIN
SET #CurrentCol = ( SELECT TOP 1 QUOTENAME(name)
FROM sys.syscolumns
WHERE object_name(id) = 'Order'
AND name <> 'PromoCode'
AND QUOTENAME(name) > #CurrentCol
ORDER BY name)
IF #CurrentCol IS NULL Break;
SET #InsertSQL = #InsertSQL + #CurrentCol + ', '
SET #SelectSQL = #SelectSQL + #CurrentCol + ', '
END
--Finish and concatenate the strings
SET #InsertSQL = #InsertSQL + 'PromoCode) '
SET #SelectSQL = #SelectSQL + '''' + #PromoCode + '''' + ' FROM INSERTED'
DECLARE #MasterSQL nvarchar(2000)
SET #MasterSQL = #InsertSQL + #SelectSQL
EXEC (#MasterSQL)
END
BTW - "order" is a poor choice for a table name - it's also a reserved word in SQL. Try Orders or OrderHeader.

Select columns with NULL values only

How do I select all the columns in a table that only contain NULL values for all the rows? I'm using MS SQL Server 2005. I'm trying to find out which columns are not used in the table so I can delete them.
Here is the sql 2005 or later version: Replace ADDR_Address with your tablename.
declare #col varchar(255), #cmd varchar(max)
DECLARE getinfo cursor for
SELECT c.name FROM sys.tables t JOIN sys.columns c ON t.Object_ID = c.Object_ID
WHERE t.Name = 'ADDR_Address'
OPEN getinfo
FETCH NEXT FROM getinfo into #col
WHILE ##FETCH_STATUS = 0
BEGIN
SELECT #cmd = 'IF NOT EXISTS (SELECT top 1 * FROM ADDR_Address WHERE [' + #col + '] IS NOT NULL) BEGIN print ''' + #col + ''' end'
EXEC(#cmd)
FETCH NEXT FROM getinfo into #col
END
CLOSE getinfo
DEALLOCATE getinfo
SELECT cols
FROM table
WHERE cols IS NULL
This should give you a list of all columns in the table "Person" that has only NULL-values. You will get the results as multiple result-sets, which are either empty or contains the name of a single column. You need to replace "Person" in two places to use it with another table.
DECLARE crs CURSOR LOCAL FAST_FORWARD FOR SELECT name FROM syscolumns WHERE id=OBJECT_ID('Person')
OPEN crs
DECLARE #name sysname
FETCH NEXT FROM crs INTO #name
WHILE ##FETCH_STATUS = 0
BEGIN
EXEC('SELECT ''' + #name + ''' WHERE NOT EXISTS (SELECT * FROM Person WHERE ' + #name + ' IS NOT NULL)')
FETCH NEXT FROM crs INTO #name
END
CLOSE crs
DEALLOCATE crs
Or did you want to just see if a column only has NULL values (and, thus, is probably unused)?
Further clarification of the question might help.
EDIT:
Ok.. here's some really rough code to get you going...
SET NOCOUNT ON
DECLARE #TableName Varchar(100)
SET #TableName='YourTableName'
CREATE TABLE #NullColumns (ColumnName Varchar(100), OnlyNulls BIT)
INSERT INTO #NullColumns (ColumnName, OnlyNulls) SELECT c.name, 0 FROM syscolumns c INNER JOIN sysobjects o ON c.id = o.id AND o.name = #TableName AND o.xtype = 'U'
DECLARE #DynamicSQL AS Nvarchar(2000)
DECLARE #ColumnName Varchar(100)
DECLARE #RC INT
SELECT TOP 1 #ColumnName = ColumnName FROM #NullColumns WHERE OnlyNulls=0
WHILE ##ROWCOUNT > 0
BEGIN
SET #RC=0
SET #DynamicSQL = 'SELECT TOP 1 1 As HasNonNulls FROM ' + #TableName + ' (nolock) WHERE ''' + #ColumnName + ''' IS NOT NULL'
EXEC sp_executesql #DynamicSQL
set #RC=##rowcount
IF #RC=1
BEGIN
SET #DynamicSQL = 'UPDATE #NullColumns SET OnlyNulls=1 WHERE ColumnName=''' + #ColumnName + ''''
EXEC sp_executesql #DynamicSQL
END
ELSE
BEGIN
SET #DynamicSQL = 'DELETE FROM #NullColumns WHERE ColumnName=''' + #ColumnName+ ''''
EXEC sp_executesql #DynamicSQL
END
SELECT TOP 1 #ColumnName = ColumnName FROM #NullColumns WHERE OnlyNulls=0
END
SELECT * FROM #NullColumns
DROP TABLE #NullColumns
SET NOCOUNT OFF
Yes, there are easier ways, but I have a meeting to go to right now. Good luck!
Here is an updated version of Bryan's query for 2008 and later. It uses INFORMATION_SCHEMA.COLUMNS, adds variables for the table schema and table name. The column data type was added to the output. Including the column data type helps when looking for a column of a particular data type. I didn't added the column widths or anything.
For output the RAISERROR ... WITH NOWAIT is used so text will display immediately instead of all at once (for the most part) at the end like PRINT does.
SET NOCOUNT ON;
DECLARE
#ColumnName sysname
,#DataType nvarchar(128)
,#cmd nvarchar(max)
,#TableSchema nvarchar(128) = 'dbo'
,#TableName sysname = 'TableName';
DECLARE getinfo CURSOR FOR
SELECT
c.COLUMN_NAME
,c.DATA_TYPE
FROM
INFORMATION_SCHEMA.COLUMNS AS c
WHERE
c.TABLE_SCHEMA = #TableSchema
AND c.TABLE_NAME = #TableName;
OPEN getinfo;
FETCH NEXT FROM getinfo INTO #ColumnName, #DataType;
WHILE ##FETCH_STATUS = 0
BEGIN
SET #cmd = N'IF NOT EXISTS (SELECT * FROM ' + #TableSchema + N'.' + #TableName + N' WHERE [' + #ColumnName + N'] IS NOT NULL) RAISERROR(''' + #ColumnName + N' (' + #DataType + N')'', 0, 0) WITH NOWAIT;';
EXECUTE (#cmd);
FETCH NEXT FROM getinfo INTO #ColumnName, #DataType;
END;
CLOSE getinfo;
DEALLOCATE getinfo;
You can do:
select
count(<columnName>)
from
<tableName>
If the count returns 0 that means that all rows in that column all NULL (or there is no rows at all in the table)
can be changed to
select
case(count(<columnName>)) when 0 then 'Nulls Only' else 'Some Values' end
from
<tableName>
If you want to automate it you can use system tables to iterate the column names in the table you are interested in
If you need to list all rows where all the column values are NULL, then i'd use the COLLATE function. This takes a list of values and returns the first non-null value. If you add all the column names to the list, then use IS NULL, you should get all the rows containing only nulls.
SELECT * FROM MyTable WHERE COLLATE(Col1, Col2, Col3, Col4......) IS NULL
You shouldn't really have any tables with ALL the columns null, as this means you don't have a primary key (not allowed to be null). Not having a primary key is something to be avoided; this breaks the first normal form.
Try this -
DECLARE #table VARCHAR(100) = 'dbo.table'
DECLARE #sql NVARCHAR(MAX) = ''
SELECT #sql = #sql + 'IF NOT EXISTS(SELECT 1 FROM ' + #table + ' WHERE ' + c.name + ' IS NOT NULL) PRINT ''' + c.name + ''''
FROM sys.objects o
JOIN sys.columns c ON o.[object_id] = c.[object_id]
WHERE o.[type] = 'U'
AND o.[object_id] = OBJECT_ID(#table)
AND c.is_nullable = 1
EXEC(#sql)
Not actually sure about 2005, but 2008 ate it:
USE [DATABASE_NAME] -- !
GO
DECLARE #SQL NVARCHAR(MAX)
DECLARE #TableName VARCHAR(255)
SET #TableName = 'TABLE_NAME' -- !
SELECT #SQL =
(
SELECT
CHAR(10)
+'DELETE FROM ['+t1.TABLE_CATALOG+'].['+t1.TABLE_SCHEMA+'].['+t1.TABLE_NAME+'] WHERE '
+(
SELECT
CASE t2.ORDINAL_POSITION
WHEN (SELECT MIN(t3.ORDINAL_POSITION) FROM INFORMATION_SCHEMA.COLUMNS t3 WHERE t3.TABLE_NAME=t2.TABLE_NAME) THEN ''
ELSE 'AND '
END
+'['+COLUMN_NAME+'] IS NULL' AS 'data()'
FROM INFORMATION_SCHEMA.COLUMNS t2 WHERE t2.TABLE_NAME=t1.TABLE_NAME FOR XML PATH('')
) AS 'data()'
FROM INFORMATION_SCHEMA.TABLES t1 WHERE t1.TABLE_NAME = #TableName FOR XML PATH('')
)
SELECT #SQL -- EXEC(#SQL)
Here I have created a script for any kind of SQL table. please copy this stored procedure and create this on your Environment and run this stored procedure with your Table.
exec [dbo].[SP_RemoveNullValues] 'Your_Table_Name'
stored procedure
GO
/****** Object: StoredProcedure [dbo].[SP_RemoveNullValues] Script Date: 09/09/2019 11:26:53 AM ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
-- akila liyanaarachchi
Create procedure [dbo].[SP_RemoveNullValues](#PTableName Varchar(50) ) as
begin
DECLARE Cussor CURSOR FOR
SELECT COLUMN_NAME,TABLE_NAME,DATA_TYPE
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = #PTableName
OPEN Cussor;
Declare #ColumnName Varchar(50)
Declare #TableName Varchar(50)
Declare #DataType Varchar(50)
Declare #Flage int
FETCH NEXT FROM Cussor INTO #ColumnName,#TableName,#DataType
WHILE ##FETCH_STATUS = 0
BEGIN
set #Flage=0
If(#DataType in('bigint','numeric','bit','smallint','decimal','smallmoney','int','tinyint','money','float','real'))
begin
set #Flage=1
end
If(#DataType in('date','atetimeoffset','datetime2','smalldatetime','datetime','time'))
begin
set #Flage=2
end
If(#DataType in('char','varchar','text','nchar','nvarchar','ntext'))
begin
set #Flage=3
end
If(#DataType in('binary','varbinary'))
begin
set #Flage=4
end
DECLARE #SQL VARCHAR(MAX)
if (#Flage in(1,4))
begin
SET #SQL =' update ['+#TableName+'] set ['+#ColumnName+']=0 where ['+#ColumnName+'] is null'
end
if (#Flage =3)
begin
SET #SQL =' update ['+#TableName+'] set ['+#ColumnName+'] = '''' where ['+#ColumnName+'] is null '
end
if (#Flage =2)
begin
SET #SQL =' update ['+#TableName+'] set ['+#ColumnName+'] ='+'''1901-01-01 00:00:00.000'''+' where ['+#ColumnName+'] is null '
end
EXEC(#SQL)
FETCH NEXT FROM Cussor INTO #ColumnName,#TableName,#DataType
END
CLOSE Cussor
DEALLOCATE Cussor
END
You'll have to loop over the set of columns and check each one. You should be able to get a list of all columns with a DESCRIBE table command.
Pseudo-code:
foreach $column ($cols) {
query("SELECT count(*) FROM table WHERE $column IS NOT NULL")
if($result is zero) {
# $column contains only null values"
push #onlyNullColumns, $column;
} else {
# $column contains non-null values
}
}
return #onlyNullColumns;
I know this seems a little counterintuitive but SQL does not provide a native method of selecting columns, only rows.
I would also recommend to search for fields which all have the same value, not just NULL.
That is, for each column in each table do the query:
SELECT COUNT(DISTINCT field) FROM tableName
and concentrate on those which return 1 as a result.
SELECT t.column_name
FROM user_tab_columns t
WHERE t.nullable = 'Y' AND t.table_name = 'table name here' AND t.num_distinct = 0;
An updated version of 'user2466387' version, with an additional small test which can improve performance, because it's useless to test non nullable columns:
AND IS_NULLABLE = 'YES'
The full code:
SET NOCOUNT ON;
DECLARE
#ColumnName sysname
,#DataType nvarchar(128)
,#cmd nvarchar(max)
,#TableSchema nvarchar(128) = 'dbo'
,#TableName sysname = 'TableName';
DECLARE getinfo CURSOR FOR
SELECT
c.COLUMN_NAME
,c.DATA_TYPE
FROM
INFORMATION_SCHEMA.COLUMNS AS c
WHERE
c.TABLE_SCHEMA = #TableSchema
AND c.TABLE_NAME = #TableName
AND IS_NULLABLE = 'YES';
OPEN getinfo;
FETCH NEXT FROM getinfo INTO #ColumnName, #DataType;
WHILE ##FETCH_STATUS = 0
BEGIN
SET #cmd = N'IF NOT EXISTS (SELECT * FROM ' + #TableSchema + N'.' + #TableName + N' WHERE [' + #ColumnName + N'] IS NOT NULL) RAISERROR(''' + #ColumnName + N' (' + #DataType + N')'', 0, 0) WITH NOWAIT;';
EXECUTE (#cmd);
FETCH NEXT FROM getinfo INTO #ColumnName, #DataType;
END;
CLOSE getinfo;
DEALLOCATE getinfo;
You might need to clarify a bit. What are you really trying to accomplish? If you really want to find out the column names that only contain null values, then you will have to loop through the scheama and do a dynamic query based on that.
I don't know which DBMS you are using, so I'll put some pseudo-code here.
for each col
begin
#cmd = 'if not exists (select * from tablename where ' + col + ' is not null begin print ' + col + ' end'
exec(#cmd)
end

Resources