I have a table with many columns (dynamically generated columns) and a rows with USER_KEY (which has type INT).
Type of dynamic added columns is DECIMAL(15,2).
My table looks like this:
What I would like to do is get summary for each user for all columns in that row. Since there are so many of them dynamically generated, I can not hard type this. How to do this?
I also have variable #COLDEPARTMENTS, where all those dynamic columns are separated by comma, like this:
[120000003],[120000002],[140000001],[120000005],[120000021],[120000025]
I assume you are using temp tables
select *
from tempdb.INFORMATION_SCHEMA.COLUMNS
where table_name like '#MyTempTable%'
Please refer How to retrieve field names from temporary table (SQL Server 2008)
You can use the below script to get all the auto-generated columns as single row with comma separated.
Declare #tmp varchar(250)
SET #tmp = ''
select #tmp = #tmp + Column_Name + ', ' from [AdventureWorksDW2014].INFORMATION_SCHEMA.COLUMNS
where table_name like '%FactInternetSales%'
select SUBSTRING(#tmp, 0, LEN(#tmp)) as Columns
You can also store the result in variable and use that with your original table.
If you are trying to identify which columns have values and you don't want to type out the column names. Then the following should do what you want.
SELECT 'SELECT user_key, '+
+ cols.ColumnList + CHAR(10) +
+ ' FROM ' + QUOTENAME(SCHEMA_NAME(t.schema_id)) + '.' + QUOTENAME(t.name) + CHAR(10)
FROM sys.tables t
CROSS APPLY (SELECT DISTINCT STUFF(
( SELECT CHAR(10) + CHAR(9) + ', '
+ QUOTENAME(c.name) + ' = CASE WHEN ' + QUOTENAME(c.name) + ' IS NULL THEN 1 ELSE 0 END'
FROM sys.columns c
WHERE c.object_id = t.object_id
AND c.name != 'user_id'
FOR XML PATH(''),TYPE).value('(./text())[1]','VARCHAR(MAX)'),1,3,'') AS ColumnList
)cols
WHERE t.name = '{TABLE NAME}'
Related
I am working on exporting data from one environment to another environment. I want to select the list of tables which has new set of records either inserted or modified.
Database has around 200 tables and only if 10 table records are impacted since yesterday, i want to filter only those tables. Some of these tables does not have createdate table column. It is harder to identify the record difference based on plain select query to the table.
How to find the list of tables which has new set of records impacted in SQL?
And if possible only those newly impacted records from the identified tables.
I tried with this query, however this query is not returning actual tables.
select * from sysobjects where id in (
select object_id
FROM sys.dm_db_index_usage_stats
WHERE last_user_update > getdate() - 1 )
If you've not got a timestamp or something to identify newly changed records such as auditing, utilising triggers or Change Data Capture enabled on those tables, it's quiet impossible to do.
However, reading your scenario is it not possible to ignore what has changed or been modified and just simply export those 200 tables from one environment to the other and override it on the destination location?
If not, then you might be only interested in comparing data rather than identifying newly changed records to identify which tables did not match. You can do that using EXCEPT
See below example of comparing two databases with the same table names and schema creating a dynamic SQL statement column using EXCEPT from both databases on the fly and running them in a while loop; inserting each table name that was effected into a temp table.
DECLARE #Counter AS INT
, #Query AS NVARCHAR(MAX)
IF OBJECT_ID('tempdb..#CompareRecords') IS NOT NULL DROP TABLE #CompareRecords
IF OBJECT_ID('tempdb..#TablesNotMatched') IS NOT NULL DROP TABLE #TablesNotMatched
CREATE TABLE #TablesNotMatched (ObjectName NVARCHAR(200))
SELECT
ROW_NUMBER() OVER( ORDER BY (SELECT 1)) AS RowNr
, t.TABLE_CATALOG
, t.TABLE_SCHEMA
, t.TABLE_NAME
, Query = 'IF' + CHAR(13)
+ '(' + CHAR(13)
+ ' SELECT' + CHAR(13)
+ ' COUNT(*) + 1' + CHAR(13)
+ ' FROM' + CHAR(13)
+ ' (' + CHAR(13)
+ ' SELECT ' + QUOTENAME(t.TABLE_NAME, '''''') + ' AS TableName, * FROM ' + QUOTENAME(t.TABLE_CATALOG) + '.' + QUOTENAME(t.TABLE_SCHEMA) + '.' + QUOTENAME(t.TABLE_NAME) + CHAR(13)
+ ' EXCEPT' + CHAR(13)
+ ' SELECT ' + QUOTENAME(t.TABLE_NAME, '''''') + ' AS TableName, * FROM ' + QUOTENAME(t2.TABLE_CATALOG) + '.' + QUOTENAME(t.TABLE_SCHEMA) + '.' + QUOTENAME(t.TABLE_NAME) + CHAR(13)
+ ' ) AS sq' + CHAR(13)
+ ') > 1' + CHAR(13)
+ 'SELECT ' + QUOTENAME(QUOTENAME(t.TABLE_CATALOG) + '.' + QUOTENAME(t.TABLE_SCHEMA) + '.' + QUOTENAME(t.TABLE_NAME), '''''') + ' AS TableNameRecordsNotMatched'
INTO #CompareRecords
FROM <UAT_DATABASE>.INFORMATION_SCHEMA.TABLES AS t
LEFT JOIN <PROD_DATABASE>.INFORMATION_SCHEMA.TABLES AS t2 ON t.TABLE_SCHEMA = t2.TABLE_SCHEMA
AND t.TABLE_NAME = t2.TABLE_NAME
WHERE t.TABLE_TYPE = 'BASE TABLE'
SET #Counter = (SELECT MAX(RowNr) FROM #CompareRecords)
WHILE #Counter > 0
BEGIN
SET #Query = (SELECT cr.Query FROM #CompareRecords AS cr WHERE cr.RowNr = #Counter)
INSERT INTO #TablesNotMatched
EXECUTE sp_executesql #Query
SET #Counter = #Counter - 1
END
SELECT
*
FROM #TablesNotMatched
Note when using EXCEPT both tables have to have the exact same column count and type.
I hope this slightly helps.
I would like to find a way to understand if there is a relationship between two columns present in two different tables.
For example in the table [Sales].[SalesOrderHeader], I have a column SalesOrderID and in another table [Person].[EmailAddress], there is BusinessEntityID.
How can I check to see if there is a table that creates a relationship between these 2 columns? Or how can I be sure that there is not a relationship between these 2 columns?
INFORMATION_SCHEMA is what you are looking for. You can see whether or not a given column is used in a constraint by executing
SELECT * FROM INFORMATION_SCHEMA.CONSTRAINT_COLUMN_USAGE WHERE TABLE_NAME = 'Person' AND COLUMN_NAME = 'BusinessEntityID'
You will have to do some additional work to focus on your specific solution, but this is where to start.
You could do one the following to find the tables that reference [Sales].[SalesOrderHeader]:
EXEC sp_fkeys #pktable_name = N'SalesOrderHeader',#pktable_owner = N'Sales';
I apologize before hand for what follows:
create table #rels (rel_name varchar(max), matches int) declare #sql varchar(max) = '' select #sql+= char(10) + 'insert into #rels select ''' tbla + '.' + col_a + '.' + tbl_b + '.' col_b ''' colrel, count(*) from ' + tbl_a + ' join ' + tbl_b + ' on cast(' + col_a + ' as varchar(max)) = cast(' + col_b + ' as varchar(max)) from ( select a.column_name col_a, object_name(a.object_id) tbl_a, b.column_name col_b, object_name(b.object_id) tbl_b from sys.columns a cross apply sys.columns b where a.column_name <> b.column_name where a.system_type_id = b.system_type_id ) cols exec (#sql) select * from #rels where matches > 0 order by matches desc drop table #rels
I have 20 databases, each with same table but different columns.
So to make the uniform we are creating views on top of each table in a database which will contain all the columns, as there will be one application accessing all the database.
In the view, I have to write the query in such a way that if I want to alter it and add any addition column for testing I should be able to do that.
Now in below query I am altering / creating query such that it takes all the columns of that table from the database, and then I append the other columns which are not present in it.
I need to add a column which will just concatenate some of the columns
ALTER VIEW [dbo].[AIV_PARKING]
AS
SELECT
*,
Cast(NULL AS [VARCHAR](20)) [ACTCODE],
Cast(NULL AS [VARCHAR](1)) [ACTIVATEINFO],
Cast(NULL AS [VARCHAR](20)) [VEHLICNOCHECK],
Cast(NULL AS [VARCHAR](40)) [ACTIVITY],
Cast(Isnull(vehlicnocheck, '') + '|' +
Isnull(officername, '') + '|' +
Isnull(locstreet, '') + '|' +
Isnull(locsideofstreet, '') + '|' +
Isnull(loccrossstreet1, '') + '|' +
Isnull(loccrossstreet2, '') + '|'
+ Isnull(locsuburb, '') + '|'
+ Isnull(locstate, '') + '|'
+ Isnull(locpostalcode, '') + '|'
+ Isnull(loclot, '') + '|'
+ Isnull(locpostalcode, '') + '|'
+ Isnull(Cast(officerid AS VARCHAR(20)), '')
+ Isnull(officername, '') + '|'
+ Isnull(Cast (issueno AS VARCHAR(100)), '') AS NVARCHAR(max)) AS SearchText
FROM
[dbo].parking
Here I added a column called SearchText which concatenates other columns, but I get an error
Invalid column name 'VehLicNoCheck'
Is there any way I can add this column to this view?
I also tried to do to something below but I got the same error:
CAST(CASE
WHEN NOT EXISTS
(
Select 1 from INFORMATION_SCHEMA.COLUMNS
Where Column_name ='VehLicNoCheck'
and table_name='Parking'
)
THEN ''
ELSE ISNULL(VehLicNoCheck,'')
END as nvarchar(max)
)
you could create a view that normalizes the uncommon columns to rows, where the values for the common columns are just repeated, e.g:
select id, col, value from parking
unpivot (value for col in (actcode, vehLicNoCheck, etc.)) x
the code to dynamically generate the view would be something like:
declare #sql varchar(max) = 'select id, col, value from parking unpivot (value for col in ('
select #sql += quotename(name) +',' from sys.columns where object_id=object_id('parking') and name not in ('id')
set #sql = substring(#sql, 1, len(#sql) - 1) + '))x'
exec(#sql)
this does not make sense at all.
the [ACTCODE], [ACTIVATEINFO] are all NULL value
so basically SearchText is just a string of '|||||'
you might as well, just do this
SELECT *,
CAST( NULL AS varchar) [ACTCODE],
CAST( NULL AS varchar) [ACTIVATEINFO],
CAST( NULL AS varchar) [VEHLICNOCHECK],
CAST( NULL AS varchar) [ACTIVITY],
'||||||' as SearchText
FROM [dbo].PARKING
Maybe if you can explain what are you trying to achieve here, we can point you to the right direction
EDIT :
You will need to use Dynamic SQL. You will need a list of all column names
-- declare a table variable for all the columns that you required
declare #columns table
(
id int identity,
name varchar(100)
)
-- for example these are the required columns
insert into #columns
values ('ACTCODE'), ('ACTIVATEINFO'), ('VEHLICNOCHECK'), ('ACTIVITY')
-- The Query to create the view
declare #sql nvarchar(max)
select #sql = N'CREATE VIEW [AIV_PARKING] AS' + char(13) + 'SELECT' + char(13)
select #sql = #sql
+ case when t.name is not null
then quotename(c.name) + ','
else 'CAST (NULL AS VARCHAR(10)) AS ' + quotename(c.name) + ','
end
+ char(13)
from #columns c
left join sys.columns t on c.name = t.name
and t.object_id = object_id('PARKING')
order by c.id
select #sql = #sql
+ case when t.name is not null
then 'ISNULL(' + quotename(c.name) + ', '''')'
else ''
end
+ ' + ''|'''
+ ' + '
from #columns c
left join sys.columns t on c.name = t.name
and t.object_id = object_id('PARKING')
order by c.id
select #sql = left(#sql, len(#sql) - 8) + ' AS SearchText' + char(13)
+ 'FROM PARKING'
-- print out to view the complete create view statement
print #sql
-- execute it
exec sp_executesql #sql
Using a single select query, I need to get the Minimum and Maximum values of Identity Columns along with the other columns specified in the query below, for all tables in a given database.
This is what I've been able to code to get a list of tables and their identity columns:
Select so.name as TableName
, sic.name as ColumnName
, i.Rows Count_NumberOfRecords
, IDENT_CURRENT(so.name)+IDENT_INCR(so.name) as NextSeedValue
from sys.identity_columns sic
inner join sys.objects so on sic.object_id = so.object_id
inner join sys.sysindexes I ON So.OBJECT_ID = I.ID
Where so.type_desc = 'USER_TABLE' and last_value is not null and indid IN (0,1);
The query needs to get these extra columns:
MaximumValue (IdentityColumn) and MinimumValue (IdentityColumn) for each table.
You should be able to use something along these lines:
DECLARE #cmd NVARCHAR(max);
SET #cmd = '';
SELECT #cmd = #cmd + CASE WHEN (#cmd = '') THEN '' ELSE ' UNION ALL ' END + 'SELECT ''' +
QUOTENAME(s.name) + '.' + QUOTENAME(t.name) + ''' AS TableName, ''' +
QUOTENAME(c.name) + ''' AS ColumnName, MAX(' + QUOTENAME(c.name) + ') AS MaxID, MIN(' +
QUOTENAME(c.name) + ') AS MinID, COALESCE(IDENT_CURRENT(''' + QUOTENAME(s.name) + '.' +
QUOTENAME(t.name) + '''),0) + COALESCE(IDENT_INCR(''' + QUOTENAME(s.name) + '.' +
QUOTENAME(t.name) + '''),0) AS NextValue FROM ' + QUOTENAME(s.name) + '.' + QUOTENAME(t.name)
FROM sys.tables t
INNER JOIN sys.columns c ON t.object_id = c.object_id
INNER JOIN sys.schemas s on t.schema_id = s.schema_id
WHERE c.is_identity = 1
SELECT #cmd; /* Shows the dynamic query generated, not necessary */
EXEC sp_executesql #cmd;
The query uses dynamic SQL to construct a UNION query that gathers the Table Name, Column Name, and Min and Max ID values currently in every table that has an IDENTITY field.
You could quite easily modify this to show the columns in the format you want, along with the other columns you mention in your question.
I've edited the query above to include the "NextValue" field, however I agree with #AaronBertrand in that this value is of little use, since in a busy system it will most certainly be wrong immediately (or shortly thereafter) once the query executes.
I NEVER do complicated stuff in SQL - until now...
I have a database with over 2000 tables, each table has about 200 columns.
I need to get a list of all the columns in one of those tables that are populated at least 1 time.
I can get a list of all the columns like this:
SELECT [name] AS [Column name]
FROM syscolumns with (nolock)
WHERE id = (SELECT id FROM sysobjects where name like 'DOCSDB_TDCCINS')
But I need only the columns that are populated 1 or more times.
Any help would be appreciated.
Here is how I would do it, first run this:
SELECT 'SELECT '''+syscolumns.name+''' FROM '+sysobjects.name+' HAVING COUNT('+syscolumns.name+') > 0'
FROM syscolumns with (nolock)
JOIN sysobjects with (nolock) ON syscolumns.id = sysobjects.id
WHERE syscolumns.id = (SELECT id FROM sysobjects where name like 'Email')
Copy all the select statements and run them.
This will give you a list of the column names without nulls.
(nb I did not test because I don't have an SQL server available right now, so I could have a typo)
It may be also be useful to count the non-null instances, obviously 0 or not 0 was your initial question, and counting the instances versus exists not/exists will be slower.
select 'union select ''' + Column_Name + ''',count(*)'
+ ' from ' + table_name
+ ' where ' + column_name + ' is not null'
from
(
select * from information_schema.columns with (nolock)
where Is_Nullable = 'YES'
AND Table_Name like 'DOCSDB_TDCCINS'
) DD
Then remove the superfluous leading 'union' and run the query
A different idea is to create a dynamic unpivot for every table.
Declare #q NVarchar(MAX) = NULL
;With D AS (
SELECT TABLE_SCHEMA
, TABLE_NAME
, STUFF((SELECT ', ' + QUOTENAME(ci.COLUMN_NAME)
FROM INFORMATION_SCHEMA.COLUMNS ci
WHERE (ci.TABLE_NAME = c.TABLE_NAME)
AND (ci.TABLE_SCHEMA = c.TABLE_SCHEMA)
FOR XML PATH(''),TYPE).value('.','NVARCHAR(MAX)')
,1,2,'') AS _Cols
, STUFF((SELECT ', Count(' + QUOTENAME(ci.COLUMN_NAME) + ') '
+ QUOTENAME(ci.COLUMN_NAME)
FROM INFORMATION_SCHEMA.COLUMNS ci
WHERE (ci.TABLE_NAME = c.TABLE_NAME)
AND (ci.TABLE_SCHEMA = c.TABLE_SCHEMA)
FOR XML PATH(''),TYPE).value('.','NVARCHAR(MAX)')
,1,2,'') AS _ColsCount
FROM INFORMATION_SCHEMA.COLUMNS c
GROUP BY TABLE_SCHEMA, TABLE_NAME
)
SELECT #q = COALESCE(#q + ' UNION ALL ', '') + '
SELECT ''' + TABLE_SCHEMA + ''' _Schema, ''' + TABLE_NAME + ''' _Table, _Column
FROM (SELECT ' + _ColsCount + ' from ' + TABLE_SCHEMA + '.' + TABLE_NAME + ') x
UNPIVOT
(_Count FOR _Column IN (' + _Cols + ')) u
WHERE _Count > 0'
FROM D
exec sp_executesql #q
In the CTE _Cols returns the comma separated quoted name of the columns of the table, while _ColsCount returns the same list with the COUNT function, for example for a table of mine a row of D is
TABLE_SCHEMA | TABLE_NAME | _Cols | _ColsCount
------------- ----------------- ------------------------------ -----------------------------------------------------------------------------
dbo | AnnualInterests | [Product_ID], [Rate], [Term] | Count([Product_ID]) [Product_ID], Count([Rate]) [Rate], Count([Term]) [Term]
while the main query trasform this line in the UNPIVOT to return the columns in rows
SELECT 'dbo' _Schema, 'AnnualInterests' _Table, _Column
FROM (SELECT Count([Product_ID]) [Product_ID], Count([Term]) [Term]
, Count([Rate]) [Rate] from dbo.AnnualInterests) x
UNPIVOT
(_Count FOR _Column IN ([Product_ID], [Term], [Rate])
WHERE _Count > 0
using the string variable concatenation and sp_executesql to run the string complete the script.
Hope you can achieve this by a simple alteration on your code like
SELECT [name] AS [Column name]
FROM syscolumns with (nolock)
WHERE id = (SELECT id FROM sysobjects where name like 'DOCSDB_TDCCINS')
and (select count(*) from DOCSDB_TDCCINS)>0