I am using the following query to insert in the respective historical table changes occurred to a given table. I am executing the same query simultaneously for multiple tables in python (changing the table name and database). None of the historical tables have foreign keys. But some of the executions end up in deadlock. Each table have assign a unique historical table. I am not sure how to solve the issue. Is it because I use a variable table with the same name in all the procedures?
declare #name_tab table (name_column varchar(200),
dtype varchar(200))
declare #columns varchar(max)
declare #query varchar(max)
declare #database varchar(200)
declare #table_name varchar(200)
set #database = '%s'
set #table_name = '%s'
insert into #name_tab
select c.name as name_column,
t.name as dtype
from sys.all_columns c
INNER JOIN sys.types t
ON t.system_type_id = c.system_type_id
where OBJECT_NAME(c.object_id) = #table_name
set #columns= stuff((select ','+name_column from #name_tab FOR XML PATH('')),1, 1, '')
set #query= 'insert into ' +#database+'..'+'HISTORY_'+#table_name+' select super_q.* from' +
'(select cast (GETDATE() as smalldatetime) as TIME_MODIFIED, new_info.* from '+
'(SELECT ' + #columns + ' From '+#database+'..'+#table_name +
' except ' +
'SELECT ' + #columns + ' From '+#database+'..'+'HISTORY_'+#table_name + ') new_info) as super_q'
execute(#query)
I got this sample from system_health
It appears that some concurrent process is altering or creating a table at the same time. The deadlock XML should contain additional details about what's going on.
But whatever the actual cause, the solution is simple. Use your scripting above to generate the trigger bodies in static SQL so you don't have to query the catalog for every insert.
Create a procedure in your database called, say, admin.GenerateHistoryTables and one called admin.GenerateHistoryTriggers and run those ahead of time to install the history tables and wire up the triggers.
Or stop re-inventing the wheel and use Change Data Capture or Temporal Tables.
Related
Given a Database server on which I only have read access to the Master DB, I need to scan all databases on the server for tables that contain "SMTP Mail Setup" in their name. I also need to know the value of the field "SMTP Server" within each of those tables.
I've been able to cobble together the following which lists the Database and Table names where the data I need is stored.
EXEC sp_MSforeachdb 'USE [?] SELECT TABLE_CATALOG as DB_Name, Table_Name FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME LIKE ''%SMTP Mail Setup%'''
I'm stuck now as I can't seem to figure out how to pull the field "SMTP Server" from the given tables. Is there a better way to approach this?
You will need to generate and execute dynamic SQL based on the results of the first query.
Try the following (somewhat over-engineered) code:
DECLARE #TableNamePattern sysname = '%SMTP Mail Setup%'
DECLARE #ColumnName sysname = 'SMTP Server'
IF OBJECT_ID('TempDb..#SelectedTables') IS NOT NULL DROP TABLE #SelectedTables
CREATE TABLE #SelectedTables (DB_Name sysname, Table_Name sysname)
DECLARE #SqlTemplate1 VARCHAR(MAX) = '
USE [?]
INSERT #SelectedTables
SELECT T.TABLE_CATALOG as DB_Name, T.TABLE_NAME
FROM INFORMATION_SCHEMA.TABLES T
JOIN INFORMATION_SCHEMA.COLUMNS C
ON C.TABLE_CATALOG = T.TABLE_CATALOG
AND C.TABLE_SCHEMA = T.TABLE_SCHEMA
AND C.TABLE_NAME = T.TABLE_NAME
WHERE T.TABLE_TYPE = ''BASE TABLE''
AND T.TABLE_NAME LIKE <TableNamePattern>
AND C.COLUMN_NAME = <ColumnName>
'
DECLARE #Sql1 VARCHAR(MAX) =
REPLACE(REPLACE(
#SqlTemplate1
, '<TableNamePattern>', QUOTENAME(#TableNamePattern, ''''))
, '<ColumnName>', QUOTENAME(#ColumnName, ''''))
EXEC sp_MSforeachdb #Sql1
SELECT * FROM #SelectedTables ORDER BY DB_Name, Table_Name
DECLARE #SqlTemplate2 VARCHAR(MAX) = 'UNION ALL
SELECT <DB_NAME_Text> AS DB_NAME, <Table_Name_Text> AS Table_Name, <Column_Name>
FROM <DB_NAME>..<Table_Name>
'
DECLARE #Sql2 VARCHAR(MAX) = STUFF((
SELECT REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(
#SqlTemplate2
, '<DB_NAME_Text>', QUOTENAME(T.DB_NAME, ''''))
, '<Table_Name_Text>', QUOTENAME(T.Table_Name, ''''))
, '<DB_NAME>', QUOTENAME(T.DB_NAME))
, '<Table_Name>', QUOTENAME(T.Table_Name))
, '<Column_Name>', QUOTENAME(#ColumnName))
FROM #SelectedTables T
ORDER BY T.DB_NAME, T.Table_Name
FOR XML PATH(''),TYPE
).value('text()[1]','nvarchar(max)')
, 1, 9, '') -- Remove initial UNION ALL
SET #Sql2 = #Sql2 + '
ORDER BY 1, 2, 3' -- Lazy way of referencing columns
PRINT #Sql2 -- Might be truncated
EXEC (#Sql2)
DROP TABLE #SelectedTables
I added checks to ensure that the column is defined in the selected table and that the table is a true table ('BASE TABLE') and not a view. The sql templates are run through a series a replace functions that insert the properly quoted and escaped object names. The first template is an expanded version of your original executed sql. The second is used to generate a series or selects for each table.
The FOR XML PATH(''),TYPE concatenates all of the generated selects into a single XML string, and the .value() at the end reliably extracts that text avoiding any XML encoding artifacts. Newer SQL Server versions support a STRING_AGG() function that can be used instead, but the code I had on hand was already using FOR XML.
Each query starts with UNION ALL so that all results display in a combined grid. The STUFF(..., 1, 11, '') strips off the leading UNION ALL.
Finally the resulting SQL is printed and executed.
I have a system that takes in Revit models and loads all the data in the model to a 2016 SQL Server. Unfortunately, the way the system works it created a new database for each model that is loaded. All the databases start with an identical schema because there is a template database that the system uses to build any new ones.
I need to build a view that can query data from all databases on the server but can automatically add new databases as they are created. The table names and associated columns will be identical across all databases, including data types.
Is there a way to pull a list of current database names using:
SELECT [name] FROM sys.databases
and then use the results to UNION the results from a basic SELECT query like this:
SELECT
[col1]
,[col2]
,[col3]
FROM [database].[dbo].[table]
Somehow replace the [database] part with the results of the sys.databases query?
The goal would be for the results to look as if I did this:
SELECT
[col1]
,[col2]
,[col3]
FROM [database1].[dbo].[table]
UNION
SELECT
[col1]
,[col2]
,[col3]
FROM [database2].[dbo].[table]
but dynamically for all databases on the server and without future management from me.
Thanks in advance for the assistance!
***Added Info: A couple suggestions using STRING_AGG have been made, but that function is not available in 2016.
Try this. It will automatically detect and include new databases with the specified table name. If a database is dropped it will automatically exclude it.
I updated the TSQL. STRING_AGG concatenates the string with each database. Without it it only returns the last database. STRING_AGG is more secure than += which also concatenates. I changed the code so it generates and executes the query. In SQL 2019 the query is all in one line using +=. I don't have SQL 2016. It may format it better in SQL 2016. You can uncomment --SELECT #SQL3 to see what the query looks like. Please mark as answer if this is what you need.
DECLARE #TblName TABLE
(
TblName VARCHAR(100)
)
Declare #SQL VARCHAR(MAX),
#SQL3 VARCHAR(MAX),
#DBName VARCHAR(50),
#Count Int,
#LoopCount Int
Declare #SQL2 VARCHAR(MAX) = ''
Select Identity(int,1,1) ID, name AS DBName into #Temp from sys.databases
Select #Count = ##RowCount
Set #LoopCount = 1
While #LoopCount <= #Count
Begin
SET #DBName = (SELECT DBName FROM #Temp Where ID = #LoopCount)
SET #SQL =
' USE ' + #DBName +
' SELECT TABLE_CATALOG FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = ''table'''
INSERT INTO #TblName (TblName)
EXEC (#SQL)
Set #LoopCount=#LoopCount + 1
End
SELECT #SQL2 +=
' SELECT ' + char(10) +
' [col1] ' + char(10) +
' ,[col2] ' + char(10) +
' ,[col3] ' + char(10) +
' FROM [' + TblName + '].[dbo].[table] ' + char(10) +
' UNION '
FROM #TblName
DROP TABLE #Temp
SET #SQL3 = (SELECT SUBSTRING(#SQL2, 1, LEN(#SQL2) - 5))
--SELECT #SQL3
EXEC (#SQL3)
I am consistently running a report and creating tables for this report. Now other users are running thsi report as well. So I need users to be able to run stored procedure simultaniously without worry of overwriting tables. I tried using a simple temp table but I need the temporary table to work through out two "functions." One dynamic sql statement that creates a table and one dynamic sql statment thats table driven.
My primary issue is I want the table driven piece of code to be able to see the global temporary table variable but it does not. Is there a work around for this while still using temporary tables? is there a way to run both dynamic sql statements at once so the other type of temp table would work?
Any advice in the right direction is helpful. Thank you.
DECLARE #TmpGlobalTable varchar(255) = 'SomeText_' + convert(varchar(36),NEWID())
SELECT #SQL = #SQL +'
SELECT IDENTITY(INT) as idcol, date, Desc As [Description]
INTO [##' + #TmpGlobalTable + ']
FROM dbo.appendix
WHERE RecordStatus = 1
and casestatement from user input
'
print(#sql)
exec(#sql)
Declare #sql1 varchar(max) = ''
SELECT #SQL1 = #SQL1 +'
insert into dbo.'+#table+'
select ''1'', '''+date+''' as Sequence, Description as Description_color, buyer, seller, price, option
from '+#ClientTable+'
where isnull('+Seq+',9999) <= cutoffvalue
group by description , buyer, seller, price, option
'
from
[##' + #TmpGlobalTable + ']
print(#sql1)
exec(#sql1)
EXEC ('DROP TABLE [##' + #TmpGlobalTable + ']')
PRINT 'Dropped Table ' + #TmpGlobalTable
Update: I Solved this. Obviously I reinvented the wheel, but I did not immediately find the answer where I searched.
Given there is another question exactly the same as mine, but that does not answer my question, I will try to be very clear. My question is not answered because the answers do not indicate how to accomplish my task. I don't really care how I accomplish it, but there has to be a way.
I want the ability to count occurrences by level of any two discrete columns in
an arbitrary table. I want to store the results for later reference because the query takes a long time to run.
The table name and two column names should be definable.
Based on a lot of research, it appears that a function, not a procedure should be used, but I am more interested in what happens, not how it happens.
DROP FUNCTION IF EXISTS O_E_1
GO
DROP TABLE IF EXISTS TestTable
GO
CREATE FUNCTION O_E_1
(#feature NVARCHAR(128), #table NVARCHAR(128))
RETURNS TABLE
AS
RETURN
(SELECT
COUNT(DISTINCT [PersonID]) AS count_person,
#feature AS feature, [HasT2DM] AS target
FROM
dbo.#table
GROUP BY
[#feature], [HasT2DM]);
GO
SELECT *
INTO TestTable
FROM O_E_1('Diagnosis', 'PatientDiagnoses')
go
I hope that with a little bit of work, I can accomplish this.
I have a version that does this in a procedure using dynamic SQL but
unfortunately, I don't see how to save that result to a table. If someone wants to tell me how to save the results of a dynamic SELECT to a table in my schema, that would accomplish what I need.
Here is the procedure version with dynamic SQL. Also included is how I am trying to store the results into a table.
BEGIN
SET NOCOUNT ON;
DECLARE #cmd NVARCHAR(max)
set #cmd = '
(SELECT
COUNT(DISTINCT [PersonID]) AS count_person,
[' + #feature + '] AS feature, [HasT2DM] AS target
FROM
dbo.[' + #table + ']
GROUP BY
[' + #feature + '], [HasT2DM])
'
EXEC sp_executesql #cmd
END
GO
O_E_1 #feature = 'Diagnosis', #table = 'PatientDiagnoses'
SELECT *
INTO TestTable
FROM (O_E_1 #feature = 'Diagnosis', #table = 'PatientDiagnoses')
GO
I was able to code the answer I need. Here it is.
DROP PROCEDURE IF EXISTS O_E_1
GO
DROP TABLE IF EXISTS TestTable
GO
CREATE PROCEDUre O_E_1
#feature NVARCHAR(128),
#table NVARCHAR(128)
AS
BEGIN
SET NOCOUNT ON;
DECLARE #cmd NVARCHAR(max)
set #cmd = '
(SELECT
COUNT(DISTINCT [PersonID]) AS count_person,
[' + #feature + '] AS feature, [HasT2DM] AS target
FROM
dbo.[' + #table + ']
GROUP BY
[' + #feature + '], [HasT2DM])
'
EXEC sp_executesql #cmd
END
GO
DROP TABLe IF EXISTS RESULT
CREATE TABLE Result
(count_person numeric,
feature varchar(128),
target varchar(128)
)
INSERT Result EXEC O_E_1 #feature = 'Diagnosis', #table = 'PatientDiagnoses'
Select TOP 100 * FROM RESULT
In a table there are like 113 columns. and there are two default records in the table, one is for unknown and another is for inapplicable. So, each column has its own default value to represent unknown and inapplicable.
I dont wanna write regular insert statement to get those two records.
so, I tried to insert each column using a cursor.
Got the names of columns for that table from information_schema.columns and tried to insert values from exact table in another location using "insert into select" statement, but the name of the columns that we get from information_schema
Declare #col_name varchar(50)
declare my_cur CURSOR for
select column_name from information_schema.columns
where table_name = 'tabl' and table_catalog = 'db'
and table_schema = 'dbo'
Fetch next from my_cur
into #col_name
while ##FETCH_STATUS = 0
BEGIN
Insert into db.dbo.tabl (***#col_name***)
select ***#col_name*** from openrowset('sqlncli', 'server=my_server; trusted_connection=yes;', db.dbo.tabl)
fetch next from my_cur into #col_name
end
close my_cur
deallocate my_cur
go
But, I did not realize that #col_name would be treated as string, rather than object (column)
Is there any work around for this case or any alternative solution.
I think that getting these defaults populated is the least of your problems.
I'd suggest taking a look at this: Fundamentals of Relational Database Design
And if you still want to do this, it might be better to retrieve all the defaults from the linked server, place them in a temp table, and then join to information_schema.columns to populate your table. You'll probably need to transpose the data to make it work.
You will have to generate the INSERT statement as dynamic SQL and then execute it
Declare #InsertStatement VarChar (Max)
SET #InsertStatement = ''
SET #InsertStatement = #InsertStatement + ' Insert into db.dbo.tabl (' + #col_name + ') '
SET #InsertStatement = #InsertStatement + ' select ' + #col_name + ' from openrowset(''sqlncli'', ''server=my_server''; '
Exec Sp_SQLExec #InsertStatement