I wonder if something like the following is possible to achieve:
ALTER TABLE dbo.[Foo]
ALTER COLUMN * NVARCHAR(500)
WHERE Columns NVARCHAR(n) < NVARCHAR(500);
you can use information_schema.columns and build a dynamic sql query to do generate query. Still you need to copy the result and run them manually.
select 'alter table '+TABLE_SCHEMA+'.'+TABLE_NAME+' alter column '+COLUMN_NAME+' NVARCHAR(500)'
from INFORMATION_SCHEMA.columns
where DATA_TYPE = 'nvarchar'
and CHARACTER_MAXIMUM_LENGTH < 500
and table_name = 'foo'
and TABLE_SCHEMA = 'dbo'
edited below
use the below code to include null/not null
select 'alter table ['+TABLE_SCHEMA+'].['+TABLE_NAME+'] alter column ['+COLUMN_NAME+'] NVARCHAR(500) '+case when IS_NULLABLE = 'YES' then 'NULL' else 'NOT NULL' end
from INFORMATION_SCHEMA.columns
where DATA_TYPE = 'nvarchar'
and CHARACTER_MAXIMUM_LENGTH < 500
and table_name = 'foo'
and TABLE_SCHEMA = 'dbo'
I am having problems with a ETL script that is syntactically correct but does not run; it is the where clause variable which uses data from a metadata table. I've redacted all but the relevant code:
DECLARE #table VARCHAR(200) = 'MyTable'
DECLARE #w_clause NVARCHAR(MAX)
DECLARE #j_clause NVARCHAR(MAX)
-- Get non-primary key column names
DROP TABLE IF EXISTS #temp0
SELECT [Name], FieldType
INTO #temp0
FROM dbo.metadata
WHERE [Table] = #table AND
ETL_Active = 1 AND
[Name] IS NOT NULL AND
PrimaryKey <> 1
-- Get primary key column names
DROP TABLE IF EXISTS #temp1
SELECT [Name]
INTO #temp1
FROM dbo.metadata
WHERE [Table] = #table AND
ETL_Active = 1 AND
[Name] IS NOT NULL AND
PrimaryKey = 1
SELECT #w_clause = COALESCE(#w_clause+' OR ','') + 'COALESCE(prd.'+ [Name] +',' + CASE WHEN FieldType IN('char','varchar', 'nvarchar', 'nchar') THEN '''' ELSE 0 END +')' FROM #temp0
PRINT #w_clause
SELECT #j_clause = COALESCE(#j_clause+' AND ','') + 'prd.'+ [Name] + ' = ' + 'stg.' + [Name] FROM #temp1
PRINT #j_clause
The error I'm getting:
Msg 245, Level 16, State 1, Line xx
Conversion failed when converting the varchar value ')' to data type int.
This occurs when I attempt the COALESCE(prd dynamic SQL assignment to the variable for the WHERE clause. This has me stomped so any help is appreciated. Thanks.
You should use CONCAT instead of + for string concatenation. With + you have to cast both arguments to string, and replace NULL with ''. CONCAT does both for you.
That should be:
SELECT #w_clause = COALESCE(#w_clause+' OR ','') + 'COALESCE(prd.'+ [Name] +',' + CASE
WHEN FieldType IN('char','varchar', 'nvarchar', 'nchar') THEN '''' ELSE '0' END +')'
or
SELECT #w_clause = concat(COALESCE(#w_clause+' OR ',''), 'COALESCE(prd.', [Name], ',', CASE WHEN FieldType IN('char','varchar', 'nvarchar', 'nchar') THEN '''' ELSE '0' END ')' )
I am trying to create a script which will alter a column only if it is not of the correct size. Something like this.
IF NOT EXISTS (SELECT COL_LENGTH('dbo.TSC701_OCT_CONTEXT', 'sql_stmt') = 1000 )
BEGIN
ALTER TABLE dbo.TSC701_OCT_CONTEXT
ALTER COLUMN sql_stmt VARCHAR(1000)
END
Sorry for the screwed up syntax
You actually do not need the SELECT statement in the IF clause.
IF (COL_LENGTH('dbo.TSC701_OCT_CONTEXT', 'sql_stmt') < 1000)
BEGIN
ALTER TABLE [Table Name]
ALTER COLUMN [Column Name] varchar(1000) null
END
Also, if your column needs to be NOT NULL or is already set to NOT NULL, you will want to switch the null from above out for NOT NULL.
If your goal is to check every column in a database, you can use the below code. Just be aware this could be very slow and could cause unintended errors and issues. I would backup whatever database you plan to use this on immediately before running it and I would also run it on a small test database or one table to see if it does what you want without taking too long.
DECLARE #TABLE_CATALOG NVARCHAR(128), #TABLE_SCHEMA NVARCHAR(128), #TABLE_NAME NVARCHAR(128), #COLUMN_NAME NVARCHAR(128), #DATA_TYPE NVARCHAR(128)
DECLARE #IS_NULLABLE NVARCHAR(3)
DECLARE #sql NVARCHAR(1000)
WHILE EXISTS (SELECT * FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_CATALOG='teoe' AND CHARACTER_MAXIMUM_LENGTH < 1000)
BEGIN
SELECT TOP 1
#TABLE_CATALOG = TABLE_CATALOG,
#TABLE_SCHEMA = TABLE_SCHEMA,
#TABLE_NAME = TABLE_NAME,
#COLUMN_NAME = COLUMN_NAME,
#IS_NULLABLE = IS_NULLABLE,
#DATA_TYPE = DATA_TYPE
FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_CATALOG='teoe'
AND CHARACTER_MAXIMUM_LENGTH < 1000
SET #sql = 'ALTER TABLE [' + #TABLE_CATALOG + '].' + #TABLE_SCHEMA + '.[' + #TABLE_NAME + '] ALTER COLUMN [' + #COLUMN_NAME + '] '+ #DATA_TYPE + '(1000) ' +
CASE WHEN #IS_NULLABLE = 'YES' THEN 'NULL' ELSE 'NOT NULL' END
EXECUTE #sql
BREAK
END
You can try it like this.
IF (SELECT CASE WHEN COL_LENGTH('dbo.TSC701_OCT_CONTEXT', 'sql_stmt') <> 1000 THEN 1 END) = 1
BEGIN ALTER TABLE dbo.TSC701_OCT_CONTEXT alter column sql_stmt varchar(1000) END
IF 1000 = (SELECT CHARACTER_MAXIMUM_LENGTH FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_CATALOG='Database Name' AND TABLE_NAME='Table Name' AND COLUMN_NAME='Column Name')
BEGIN
ALTER TABLE [Table Name]
ALTER COLUMN [Column Name] varchar(1000) null
END
Here is what I ended up with:
IF NOT EXISTS (SELECT 1 FROM sys.columns
WHERE name = 'sql_stmt' AND object_id = object_id('CONTEXT') and max_length = 1000)
BEGIN
ALTER TABLE dbo.CONTEXT ALTER COLUMN [sql_stmt] varchar (1000) NULL
END
GO
I have a SQL Server database and whenever I want to make a backup of the database, first generate a Drop and Create Script of the DB. Because by this way I can make my database in every version of SQL Server. Regardless of the version and maintenance problems of SQL.
In the Generate and Publish Scripts window of SQL Server there is an Advanced Scripting Option, as shown below:
Now, I would like a script from this window to reproduce script generator advanced options. In the other words, I want a script to make script generator of my database by my selected Advanced Options.
How can I do it?
You can use SQL Server Profiler to retrieve executing sqls in the specified instance.
[
Please refer to this How to generate SQL Table Script through query
Is this what you want?
declare #table varchar(100)
set #table = 'MyTable' -- set table name here
declare #sql table(s varchar(1000), id int identity)
-- create statement
insert into #sql(s) values ('create table [' + #table + '] (')
-- column list
insert into #sql(s)
select
' ['+column_name+'] ' +
data_type + coalesce('('+cast(character_maximum_length as varchar)+')','') + ' ' +
case when exists (
select id from syscolumns
where object_name(id)=#table
and name=column_name
and columnproperty(id,name,'IsIdentity') = 1
) then
'IDENTITY(' +
cast(ident_seed(#table) as varchar) + ',' +
cast(ident_incr(#table) as varchar) + ')'
else ''
end + ' ' +
( case when IS_NULLABLE = 'No' then 'NOT ' else '' end ) + 'NULL ' +
coalesce('DEFAULT '+COLUMN_DEFAULT,'') + ','
from information_schema.columns where table_name = #table
order by ordinal_position
-- primary key
declare #pkname varchar(100)
select #pkname = constraint_name from information_schema.table_constraints
where table_name = #table and constraint_type='PRIMARY KEY'
if ( #pkname is not null ) begin
insert into #sql(s) values(' PRIMARY KEY (')
insert into #sql(s)
select ' ['+COLUMN_NAME+'],' from information_schema.key_column_usage
where constraint_name = #pkname
order by ordinal_position
-- remove trailing comma
update #sql set s=left(s,len(s)-1) where id=##identity
insert into #sql(s) values (' )')
end
else begin
-- remove trailing comma
update #sql set s=left(s,len(s)-1) where id=##identity
end
-- closing bracket
insert into #sql(s) values( ')' )
-- result!
select s from #sql order by id
I have a large table with 500 columns and 100M rows. Based on a small sample, I believe only about 50 of the columns contain any values, and the other 450 contain only NULL values. I want to list the columns that contain no data.
On my current hardware, it would take about 24 hours to query every column (select count(1) from tab where col_n is not null)
Is there a less expensive way to determine that a column is completely empty/NULL?
What about this:
SELECT
SUM(CASE WHEN column_1 IS NOT NULL THEN 1 ELSE 0) column_1_count,
SUM(CASE WHEN column_2 IS NOT NULL THEN 1 ELSE 0) column_2_count,
...
FROM table_name
?
You can easily create this query if you use INFORMATION_SCHEMA.COLUMNS table.
EDIT:
Another idea:
SELECT MAX(column_1), MAX(column_2),..... FROM table_name
If result contains value, column is populated. It should require one table scan.
Try this one -
DDL:
IF OBJECT_ID ('dbo.test2') IS NOT NULL
DROP TABLE dbo.test2
CREATE TABLE dbo.test2
(
ID BIGINT IDENTITY(1,1) PRIMARY KEY
, Name VARCHAR(10) NOT NULL
, IsCitizen BIT NULL
, Age INT NULL
)
INSERT INTO dbo.test2 (Name, IsCitizen, Age)
VALUES
('1', 1, NULL),
('2', 0, NULL),
('3', NULL, NULL)
Query 1:
DECLARE
#TableName SYSNAME
, #ObjectID INT
, #SQL NVARCHAR(MAX)
SELECT
#TableName = 'dbo.test2'
, #ObjectID = OBJECT_ID(#TableName)
SELECT #SQL = 'SELECT' + CHAR(13) + STUFF((
SELECT CHAR(13) + ', [' + c.name + '] = ' +
CASE WHEN c.is_nullable = 0
THEN '0'
ELSE 'CASE WHEN ' + totalrows +
' = SUM(CASE WHEN [' + c.name + '] IS NULL THEN 1 ELSE 0 END) THEN 1 ELSE 0 END'
END
FROM sys.columns c WITH (NOWAIT)
CROSS JOIN (
SELECT totalrows = CAST(MIN(p.[rows]) AS VARCHAR(50))
FROM sys.partitions p
WHERE p.[object_id] = #ObjectID
AND p.index_id IN (0, 1)
) r
WHERE c.[object_id] = #ObjectID
FOR XML PATH(''), TYPE).value('.', 'NVARCHAR(MAX)'), 1, 2, ' ') + CHAR(13) + 'FROM ' + #TableName
PRINT #SQL
EXEC sys.sp_executesql #SQL
Output 1:
SELECT
[ID] = 0
, [Name] = 0
, [IsCitizen] = CASE WHEN 3 = SUM(CASE WHEN [IsCitizen] IS NULL THEN 1 ELSE 0 END) THEN 1 ELSE 0 END
, [Age] = CASE WHEN 3 = SUM(CASE WHEN [Age] IS NULL THEN 1 ELSE 0 END) THEN 1 ELSE 0 END
FROM dbo.test2
Query 2:
DECLARE
#TableName SYSNAME
, #SQL NVARCHAR(MAX)
SELECT #TableName = 'dbo.test2'
SELECT #SQL = 'SELECT' + CHAR(13) + STUFF((
SELECT CHAR(13) + ', [' + c.name + '] = ' +
CASE WHEN c.is_nullable = 0
THEN '0'
ELSE 'CASE WHEN '+
'MAX(CAST([' + c.name + '] AS CHAR(1))) IS NULL THEN 1 ELSE 0 END'
END
FROM sys.columns c WITH (NOWAIT)
WHERE c.[object_id] = OBJECT_ID(#TableName)
FOR XML PATH(''), TYPE).value('.', 'NVARCHAR(MAX)'), 1, 2, ' ') + CHAR(13) + 'FROM ' + #TableName
PRINT #SQL
EXEC sys.sp_executesql #SQL
Output 2:
SELECT
[ID] = 0
, [Name] = 0
, [IsCitizen] = CASE WHEN MAX(CAST([IsCitizen] AS CHAR(1))) IS NULL THEN 1 ELSE 0 END
, [Age] = CASE WHEN MAX(CAST([Age] AS CHAR(1))) IS NULL THEN 1 ELSE 0 END
FROM dbo.test2
Results:
ID Name IsCitizen Age
----------- ----------- ----------- -----------
0 0 0 1
Could you check if colums idexing will help you reach some performance improve
CREATE UNIQUE NONCLUSTERED INDEX IndexName ON dbo.TableName(ColumnName)
WHERE ColumnName IS NOT NULL;
GO
SQL server query to get the list of columns in a table along with Data types, NOT NULL, and PRIMARY KEY constraints
Run SQL in best answer of above questions and generate a new query like below.
Select ISNULL(column1,1), ISNULL(column2,1), ISNULL(column3,1) from table
You would not need to 'count' all of the 100M records. When you simply back out of the query with a TOP 1 as soon as you hit a column with a not-null value, would save a lot of time while providing the same information.
500 Columns?!
Ok, the right answer to your question is: normalize your table.
Here's what happening for the time being:
You don't have an index on that column so SQL Server has to do a full scan of your humongous table.
SQL Server will certainly fully read every row (it means every columns even if you're only interested in one).
And since your row are most likely over 8kb... http://msdn.microsoft.com/en-us/library/ms186981%28v=sql.105%29.aspx
Seriously, normalize your table and if needed split it horizontally (put "theme grouped" columns inside separate table, to only read them when you need them).
EDIT: You can rewrite your query like this
select count(col_n) from tab
and if you want to get all columns at once (better):
SELECT
COUNT(column_1) column_1_count,
COUNT(column_2) column_2_count,
...
FROM table_name
If most records are not null maybe you can mix some of the approach suggested (for example check only nullable fields) with this:
if exists (select * from table where field is not null)
this should speed up the search because exists stops the search as soon as condition is met, in this example a single not null record is enough to decide the status of the field.
If the field has an index this should be almost instant.
Normally adding top 1 to this query is not needed because the query optimizer knows that you do not need to retrieve all the matching records.
You can use this stored procedure to the trick You need to provide the table name you wish to query note that if you'll pass to procedure the #exec parameter = 1 it will execute the select query
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE PROCEDURE [dbo].[SP_SELECT_NON_NULL_COLUMNS] ( #tablename varchar (100)=null, #exec int =0)
AS BEGIN
SET NOCOUNT ON
IF #tablename IS NULL
RAISERROR('CANT EXECUTE THE PROC, TABLE NAME IS MISSING',16 ,1)
ELSE
BEGIN
IF OBJECT_ID('tempdb..#table') IS NOT NULL DROP TABLE #table
DECLARE #i VARCHAR (max)=''
DECLARE #sentence VARCHAR (max)=''
DECLARE #SELECT VARCHAR (max)
DECLARE #LocalTableName VARCHAR(50) = '['+#tablename+']'
CREATE TABLE #table (ColumnName VARCHAR (max))
SELECT #i+=
' IF EXISTS ( SELECT TOP 1 '+column_name+' FROM ' +#LocalTableName+' WHERE ' +column_name+
' '+'IS NOT NULL) INSERT INTO #table VALUES ('''+column_name+''');'
FROM INFORMATION_SCHEMA.COLUMNS WHERE table_name=#tablename
INSERT INTO #table
EXEC (#i)
SELECT #sentence = #sentence+' '+columnname+' ,' FROM #table
DROP TABLE #table
IF #exec=0
BEGIN
SELECT 'SELECT '+ LTRIM (left (#sentence,NULLIF(LEN (#sentence)-1,-1)))+
+' FROM ' +#LocalTableName
END
ELSE
BEGIN
SELECT #SELECT= 'SELECT '+ LTRIM (left (#sentence,NULLIF(LEN (#sentence)-1,-1)))+
+' FROM '+#LocalTableName
EXEC (#SELECT)
END
END
END
Use it like this:
EXEC [dbo].[SP_SELECT_NON_NULL_COLUMNS] 'YourTableName' , 1