I am running a dynamic sql query which loops over all tables in a large dB (1600 tables) and creates the query I need to run afterwards. After the string is created i save the string and run it. However the string concatenation is extremely slow . I need to add a lot more boilerplate code for each table but in the example below I've omitted the boilerplate code to keep it simple.
My question is how can I speed up this process of building queries via concatenation of NVARCHAR? When I run this query it gets significantly slower the more text and parameter concatenations are added. It takes over 30 mins currently but the CTE executes in 1 sec.
DECLARE #UpdateColumnsSql NVARCHAR(MAX) = '';
WITH TableColumns AS
(
SELECT
QUOTENAME(OBJECT_SCHEMA_NAME(t.object_id)) +
'.' + QUOTENAME(t.name) AS TableName
,LOWER(c.name) AS ColumnName
,ty.name AS TypeName
,c.max_length AS ColLength
FROM
sys.tables AS t
INNER JOIN
sys.columns AS c
ON
t.object_id = c.object_id
INNER JOIN
sys.types AS ty
ON
c.system_type_id = ty.system_type_id
WHERE
c.name IN
( 'Company_ID', 'Facility_ID', 'Premises_ID' )
)
SELECT
#UpdateColumnsSql = #UpdateColumnsSql +
'ALTER TABLE ' + TableColumns.TableName +
' ADD [' + TableColumns.ColumnName + '_New] ' + TypeName +
'(' + CONVERT(VARCHAR(4), ColLength) + ')' + ' NULL
SET #sql =
'''' UPDATE T
SET T.[' + TableColumns.ColumnName + '_New] = S.[NewValue]
from ' + TableColumns.TableName + ' T
inner join NewIDList S
on Company = Company_ID
where T.[' + TableColumns.ColumnName + '] = S.OldValue
AND S.[OldColumnName] = ''''''''' + TableColumns.ColumnName + ''''''''' ''''
exec(#sql);
'
FROM
TableColumns;
Related
I am working on exporting data from one environment to another environment. I want to select the list of tables which has new set of records either inserted or modified.
Database has around 200 tables and only if 10 table records are impacted since yesterday, i want to filter only those tables. Some of these tables does not have createdate table column. It is harder to identify the record difference based on plain select query to the table.
How to find the list of tables which has new set of records impacted in SQL?
And if possible only those newly impacted records from the identified tables.
I tried with this query, however this query is not returning actual tables.
select * from sysobjects where id in (
select object_id
FROM sys.dm_db_index_usage_stats
WHERE last_user_update > getdate() - 1 )
If you've not got a timestamp or something to identify newly changed records such as auditing, utilising triggers or Change Data Capture enabled on those tables, it's quiet impossible to do.
However, reading your scenario is it not possible to ignore what has changed or been modified and just simply export those 200 tables from one environment to the other and override it on the destination location?
If not, then you might be only interested in comparing data rather than identifying newly changed records to identify which tables did not match. You can do that using EXCEPT
See below example of comparing two databases with the same table names and schema creating a dynamic SQL statement column using EXCEPT from both databases on the fly and running them in a while loop; inserting each table name that was effected into a temp table.
DECLARE #Counter AS INT
, #Query AS NVARCHAR(MAX)
IF OBJECT_ID('tempdb..#CompareRecords') IS NOT NULL DROP TABLE #CompareRecords
IF OBJECT_ID('tempdb..#TablesNotMatched') IS NOT NULL DROP TABLE #TablesNotMatched
CREATE TABLE #TablesNotMatched (ObjectName NVARCHAR(200))
SELECT
ROW_NUMBER() OVER( ORDER BY (SELECT 1)) AS RowNr
, t.TABLE_CATALOG
, t.TABLE_SCHEMA
, t.TABLE_NAME
, Query = 'IF' + CHAR(13)
+ '(' + CHAR(13)
+ ' SELECT' + CHAR(13)
+ ' COUNT(*) + 1' + CHAR(13)
+ ' FROM' + CHAR(13)
+ ' (' + CHAR(13)
+ ' SELECT ' + QUOTENAME(t.TABLE_NAME, '''''') + ' AS TableName, * FROM ' + QUOTENAME(t.TABLE_CATALOG) + '.' + QUOTENAME(t.TABLE_SCHEMA) + '.' + QUOTENAME(t.TABLE_NAME) + CHAR(13)
+ ' EXCEPT' + CHAR(13)
+ ' SELECT ' + QUOTENAME(t.TABLE_NAME, '''''') + ' AS TableName, * FROM ' + QUOTENAME(t2.TABLE_CATALOG) + '.' + QUOTENAME(t.TABLE_SCHEMA) + '.' + QUOTENAME(t.TABLE_NAME) + CHAR(13)
+ ' ) AS sq' + CHAR(13)
+ ') > 1' + CHAR(13)
+ 'SELECT ' + QUOTENAME(QUOTENAME(t.TABLE_CATALOG) + '.' + QUOTENAME(t.TABLE_SCHEMA) + '.' + QUOTENAME(t.TABLE_NAME), '''''') + ' AS TableNameRecordsNotMatched'
INTO #CompareRecords
FROM <UAT_DATABASE>.INFORMATION_SCHEMA.TABLES AS t
LEFT JOIN <PROD_DATABASE>.INFORMATION_SCHEMA.TABLES AS t2 ON t.TABLE_SCHEMA = t2.TABLE_SCHEMA
AND t.TABLE_NAME = t2.TABLE_NAME
WHERE t.TABLE_TYPE = 'BASE TABLE'
SET #Counter = (SELECT MAX(RowNr) FROM #CompareRecords)
WHILE #Counter > 0
BEGIN
SET #Query = (SELECT cr.Query FROM #CompareRecords AS cr WHERE cr.RowNr = #Counter)
INSERT INTO #TablesNotMatched
EXECUTE sp_executesql #Query
SET #Counter = #Counter - 1
END
SELECT
*
FROM #TablesNotMatched
Note when using EXCEPT both tables have to have the exact same column count and type.
I hope this slightly helps.
I found this useful query to rename all my tables, indexes and constrains but I just figured out it didn't rename the columns.
SELECT 'exec sp_rename ' + '''' + NAME + '''' + ', ' + '''' + replace(NAME, 'Tb', 'Tabela') + ''''
FROM sysObjects
WHERE
NAME LIKE 'Tb%'
I know there's syscolumns but I'm not sure how to use in this case.
Question: How can I get the same result of this query but for columns instead of tables?
I appreciate your help in this. I'm using SQL Server 2012. Thanks.
You have to do a little more work:
SELECT 'exec sp_rename ' + '''' + QUOTENAME(s.name) + '.' + QUOTENAME(o.name) + '.' + QUOTENAME(c.name) + '''' + ', ' + '''' + replace(c.name, 'Col', 'Column') + ''', ''COLUMN'''
FROM sys.columns c
INNER JOIN sys.objects o ON c.object_id = o.object_id
INNER JOIN sys.schemas s ON o.schema_id = s.schema_id
WHERE c.name LIKE 'Col%'
Since you are renaming column, you must specify that third argument to sp_rename is COLUMN. You must also construct three part name in the form of [schema].[table name].[current column name] to point to the correct column.
I'm trying to retrieve all primary keys iin a fact table and then count the number of records in that fact table grouped by that id, however so far i can only seem to get all the primary keys and its talbe. i guess i need to make some kind of subquery
SELECT
tab1.name AS [FactTable],
col1.name AS [PrimaryKey]
FROM sys.indexes ind1
INNER JOIN sys.tables tab1
ON tab1.object_id = ind1.object_id
INNER JOIN sys.schemas sch1
ON tab1.schema_id = sch1.schema_id
INNER JOIN sys.columns col1
ON col1.object_id = tab1.object_id AND col1.name like '%Id'
WHERE tab1.name like 'Fact%' AND ind1.is_primary_key = 1
sample output
primaryKey countRecordsGroupedByPrimaryKey
2 4000
3 8343
4 203
1 4023
I going out on a limb and guessing that you want to count the number of each dimensionPK used in your fact table. If a fact table has references to two different dimensions, you need two different statements to count the usage by that dimension.
For the query below, provide your fact table name and schema and it should generate a count statement joining your fact and dim and grouping by the join keys.
- If a fact table has two different FK relationships, you'll get two different statements.
- If a table uses a composite PK, both key columns will be included in the join
This is complicated and I don't have any tables with multiple FKs to test it on, so please let me know if it does what you want.
DECLARE #NameOfTableWithFKs sysname = 'your fact table name',
#SchemaOfTableWithFKs sysname = 'dbo';
WITH JoinColumns
AS (SELECT QUOTENAME(OBJECT_SCHEMA_NAME(parent.object_id)) + '.' + QUOTENAME(OBJECT_NAME(parent.object_id)) AS ParentTableName,
QUOTENAME(OBJECT_SCHEMA_NAME(referenced.object_id)) + '.' + QUOTENAME(OBJECT_NAME(referenced.object_id)) AS ReferencedTableName,
QUOTENAME(OBJECT_NAME(parent.object_id)) + '.' + QUOTENAME(parent.name) + ' = ' + QUOTENAME(OBJECT_NAME(referenced.object_id)) + '.'
+ QUOTENAME(referenced.name) AS JoinColumn,
QUOTENAME(OBJECT_NAME(referenced.object_id)) + '.' + QUOTENAME(referenced.name) AS GroupingColumn
FROM sys.foreign_key_columns AS fkc
INNER JOIN sys.columns AS parent
ON parent.object_id = fkc.parent_object_id
AND parent.column_id = fkc.parent_column_id
INNER JOIN sys.columns AS referenced
ON referenced.object_id = fkc.referenced_object_id
AND referenced.column_id = fkc.referenced_column_id
WHERE OBJECT_NAME(parent.object_id) = #NameOfTableWithFKs
AND OBJECT_SCHEMA_NAME(parent.object_id) = #SchemaOfTableWithFKs
),
JoinTables
AS (SELECT QUOTENAME(OBJECT_SCHEMA_NAME(tbl.object_id)) + '.' + QUOTENAME(OBJECT_NAME(tbl.object_id)) AS ParentTableName,
QUOTENAME(OBJECT_SCHEMA_NAME(rtbl.object_id)) + '.' + QUOTENAME(OBJECT_NAME(rtbl.object_id)) AS ReferencedTableName
FROM sys.tables AS tbl
INNER JOIN sys.foreign_keys AS cstr
ON cstr.parent_object_id = tbl.object_id
INNER JOIN sys.tables AS rtbl
ON rtbl.object_id = cstr.referenced_object_id
WHERE OBJECT_NAME(tbl.object_id) = #NameOfTableWithFKs
AND OBJECT_SCHEMA_NAME(tbl.object_id) = #SchemaOfTableWithFKs
)
SELECT 'SELECT Count(*)' + ( SELECT ', ' + JC.GroupingColumn
FROM JoinColumns AS JC
WHERE JC.ParentTableName = jt.ParentTableName
AND JC.ReferencedTableName = jt.ReferencedTableName
FOR XML PATH('')
) + ' FROM ' + JT.ParentTableName + ' INNER JOIN ' + JT.ReferencedTableName + ' ON'
+ SUBSTRING(( SELECT ' AND ' + JC.JoinColumn
FROM JoinColumns AS JC
WHERE JC.ParentTableName = JT.ParentTableName
AND JC.ReferencedTableName = JT.ReferencedTableName
FOR XML PATH('')
), 5, 8000
) + ' GROUP BY ' + SUBSTRING(( SELECT ', ' + JC.GroupingColumn
FROM JoinColumns AS JC
WHERE JC.ParentTableName = JT.ParentTableName
AND JC.ReferencedTableName = JT.ReferencedTableName
FOR XML PATH('')
), 2, 8000
)
FROM JoinTables AS JT;
If I understand the question correctly you are wanting to count all the rows in some tables based on some criteria. Not really sure why you care about the primary key portion since by definition a primary key must be unique so this could still be simplified to not check for primary key but whatever.
I did remove the join to sys.columns because why does it matter the name of the column unless you want only those table named Fact% and has a column named %Id.
This should get you pretty close as I understand it.
declare #SQL nvarchar(max) = ''
select #SQL = #SQL + 'select TableName = ''' + tab1.name + ''', NumRows = count(*) from ' + QUOTENAME(sch1.name) + '.' + QUOTENAME(tab1.name) + ' UNION ALL '
FROM sys.indexes ind1
INNER JOIN sys.tables tab1
ON tab1.object_id = ind1.object_id
INNER JOIN sys.schemas sch1
ON tab1.schema_id = sch1.schema_id
--INNER JOIN sys.columns col1
-- ON col1.object_id = tab1.object_id AND col1.name like '%Id'
WHERE tab1.name like 'Fact%'
AND ind1.is_primary_key = 1
select #SQL = LEFT(#SQL, LEN(#SQL) - 10) + ' ORDER BY TableName'
select #SQL --uncomment the exec line below once you are comfortable that the dynamic sql is what you want.
--exec sp_executesql #SQL
Using a single select query, I need to get the Minimum and Maximum values of Identity Columns along with the other columns specified in the query below, for all tables in a given database.
This is what I've been able to code to get a list of tables and their identity columns:
Select so.name as TableName
, sic.name as ColumnName
, i.Rows Count_NumberOfRecords
, IDENT_CURRENT(so.name)+IDENT_INCR(so.name) as NextSeedValue
from sys.identity_columns sic
inner join sys.objects so on sic.object_id = so.object_id
inner join sys.sysindexes I ON So.OBJECT_ID = I.ID
Where so.type_desc = 'USER_TABLE' and last_value is not null and indid IN (0,1);
The query needs to get these extra columns:
MaximumValue (IdentityColumn) and MinimumValue (IdentityColumn) for each table.
You should be able to use something along these lines:
DECLARE #cmd NVARCHAR(max);
SET #cmd = '';
SELECT #cmd = #cmd + CASE WHEN (#cmd = '') THEN '' ELSE ' UNION ALL ' END + 'SELECT ''' +
QUOTENAME(s.name) + '.' + QUOTENAME(t.name) + ''' AS TableName, ''' +
QUOTENAME(c.name) + ''' AS ColumnName, MAX(' + QUOTENAME(c.name) + ') AS MaxID, MIN(' +
QUOTENAME(c.name) + ') AS MinID, COALESCE(IDENT_CURRENT(''' + QUOTENAME(s.name) + '.' +
QUOTENAME(t.name) + '''),0) + COALESCE(IDENT_INCR(''' + QUOTENAME(s.name) + '.' +
QUOTENAME(t.name) + '''),0) AS NextValue FROM ' + QUOTENAME(s.name) + '.' + QUOTENAME(t.name)
FROM sys.tables t
INNER JOIN sys.columns c ON t.object_id = c.object_id
INNER JOIN sys.schemas s on t.schema_id = s.schema_id
WHERE c.is_identity = 1
SELECT #cmd; /* Shows the dynamic query generated, not necessary */
EXEC sp_executesql #cmd;
The query uses dynamic SQL to construct a UNION query that gathers the Table Name, Column Name, and Min and Max ID values currently in every table that has an IDENTITY field.
You could quite easily modify this to show the columns in the format you want, along with the other columns you mention in your question.
I've edited the query above to include the "NextValue" field, however I agree with #AaronBertrand in that this value is of little use, since in a busy system it will most certainly be wrong immediately (or shortly thereafter) once the query executes.
I was wondering if there is an equivalent in SQL Server 2008 to Oracle's DBMS_METADATA.GET_DDL Function? You can pass this function a table name and it will return the ddl for that table so that you can use it to build a script for a schema.
I know I can go into SSMS and use that, but I would prefer to have a t-sql script that would generate the ddl for me.
Thanks,
S
I using this query for generate query but this work for 1 table :
declare #vsSQL varchar(8000)
declare #vsTableName varchar(50)
select #vsTableName = 'Customers'
select #vsSQL = 'CREATE TABLE ' + #vsTableName + char(10) + '(' + char(10)
select #vsSQL = #vsSQL + ' ' + sc.Name + ' ' +
st.Name +
case when st.Name in ('varchar','varchar','char','nchar') then '(' + cast(sc.Length as varchar) + ') ' else ' ' end +
case when sc.IsNullable = 1 then 'NULL' else 'NOT NULL' end + ',' + char(10)
from sysobjects so
join syscolumns sc on sc.id = so.id
join systypes st on st.xusertype = sc.xusertype
where so.name = #vsTableName
order by
sc.ColID
select substring(#vsSQL,1,len(#vsSQL) - 2) + char(10) + ')'
If you are looking for a TSQL solution, it is quite verbose, as [this example]¹ shows.
A shorter alternative would be using the SMO library (example)
¹ Link for this example deleted. The way Internet Archive Wayback Machine displayed an error saying that they could not display the content. And following the link to the original went someplace malicious. (Grave errors, instruction to call some number, etc.)