TRUNCATE multiple tables SQL Server 2014 - sql-server

I want to truncate multiple tables. I know that it isn't possible in the same way that DELETE will delete the rows from multiple tables.
In this question truncate multi tables IndoKnight provides the OP-designated best answer. I want to try that. However, I get a syntax error at:
TRUNCATE TABLE #tableName
To troubleshoot I tried printing the variables because when I first tried using TRUNCATE TABLE I needed to include the database name and schema (e.g. NuggetDemoDB.dbo.tablename) to get it to work. I CAN print the variable #tableList. But I CANNOT print #tableName.
DECLARE #delimiter CHAR(1),
#tableList VARCHAR(MAX),
#tableName VARCHAR(20),
#currLen INT
SET #delimiter = ','
SET #tableList = 'Employees,Products,Sales'
--PRINT #tableList
WHILE LEN(#tableList) > 0
BEGIN
SELECT #currLen =
(
CASE charindex( #delimiter, #tableList )
WHEN 0 THEN len( #tableList )
ELSE ( charindex( #delimiter, #tableList ) -1 )
END
)
SET #tableName = SUBSTRING (#tableList,1,#currLen )
--PRINT #tableName
TRUNCATE TABLE #tableName
SELECT tableList =
(
CASE ( len( #tableList ) - #currLen )
WHEN 0 THEN ''
ELSE right( #tableList, len( #tableList ) - #currLen - 1 )
END
)
END
Edit: Fixed the table list to remove the extra "Sales" from the list of tables and added "Employees".

Even thought Sales is listed twice... No harm
Declare #TableList varchar(max)
SET #tableList = 'Sales,Products,Sales'
Set #tableList = 'Truncate Table '+replace(#tablelist,',',';Truncate Table ')+';'
Print #TableList
--Exec(#tablelist) --<< If you are TRULY comfortable with the results
Returns
Truncate Table Sales;Truncate Table Products;Truncate Table Sales

First and foremost, you may want to consider spending a little energy to come up with a SQL implementation for splitting a string into rows, e.g. Split, List, etc. This will prove to be helpful not only for this exercise, but for many others. Then this post is not about how to turn a comma separated list into rows and we can then concentrate on the dynamic SQL needed in order to do what is needed.
Example
The below example assumes that you have a function named List to take care of transposing the comma separated list into rows.
declare
#TableList varchar(max) = 'Sales, Products, Sales';
declare
#Sql varchar(max) = (
select distinct 'truncate table ' + name + ';'
from List(#TableList)
for xml path(''));
exec (#Sql);
One last thing about truncate of delete
Truncate will not work if you are truncating data where there is a foreign key relationship to another table.
You will get something like the below error.
Msg 4712, Level 16, State 1, Line 19
Cannot truncate table 'Something' because it is being referenced by a FOREIGN KEY constraint.

Below is an example that uses a table variable instead of delimited list. If the source of your table list is already in a table, you could tweak this script to use that as the source instead. Note that the extra Sales table is redundant (gleaned from the script your question) and can be removed. The table names can be database and/or schema qualified if desired.
DECLARE #tableList TABLE(TableName nvarchar(393));
DECLARE #TruncateTableBatch nvarchar(MAX);
INSERT INTO #tableList VALUES
(N'Sales')
, (N'Products')
, (N'Sales');
SET #TruncateTableBatch = (SELECT N'TRUNCATE TABLE ' + TableName + N'
'
FROM #tableList
FOR XML PATH(''), TYPE).value('.', 'nvarchar(MAX)');
--PRINT #SQL;
EXECUTE(#TruncateTableBatch);

What about something like:
exec sp_msforeachtable
#command1 ='truncate table ?'
,#whereand = ' and object_id In (select object_id from sys.objects where name in ("sales", "products")'
Have not tested it yet. But it might give a useful hint.

Related

SQL: How to insert the result of a stored procedure to a temp table [duplicate]

How do I do a SELECT * INTO [temp table] FROM [stored procedure]? Not FROM [Table] and without defining [temp table]?
Select all data from BusinessLine into tmpBusLine works fine.
select *
into tmpBusLine
from BusinessLine
I am trying the same, but using a stored procedure that returns data, is not quite the same.
select *
into tmpBusLine
from
exec getBusinessLineHistory '16 Mar 2009'
Output message:
Msg 156, Level 15, State 1, Line 2
Incorrect syntax near the keyword
'exec'.
I have read several examples of creating a temporary table with the same structure as the output stored procedure, which works fine, but it would be nice to not supply any columns.
You can use OPENROWSET for this. Have a look. I've also included the sp_configure code to enable Ad Hoc Distributed Queries, in case it isn't already enabled.
CREATE PROC getBusinessLineHistory
AS
BEGIN
SELECT * FROM sys.databases
END
GO
sp_configure 'Show Advanced Options', 1
GO
RECONFIGURE
GO
sp_configure 'Ad Hoc Distributed Queries', 1
GO
RECONFIGURE
GO
SELECT * INTO #MyTempTable FROM OPENROWSET('SQLNCLI', 'Server=(local)\SQL2008;Trusted_Connection=yes;',
'EXEC getBusinessLineHistory')
SELECT * FROM #MyTempTable
If you want to do it without first declaring the temporary table, you could try creating a user-defined function rather than a stored procedure and make that user-defined function return a table. Alternatively, if you want to use the stored procedure, try something like this:
CREATE TABLE #tmpBus
(
COL1 INT,
COL2 INT
)
INSERT INTO #tmpBus
Exec SpGetRecords 'Params'
In SQL Server 2005 you can use INSERT INTO ... EXEC to insert the result of a stored procedure into a table. From MSDN's INSERT documentation (for SQL Server 2000, in fact):
--INSERT...EXECUTE procedure example
INSERT author_sales EXECUTE get_author_sales
This is an answer to a slightly modified version of your question. If you can abandon the use of a stored procedure for a user-defined function, you can use an inline table-valued user-defined function. This is essentially a stored procedure (will take parameters) that returns a table as a result set; and therefore will place nicely with an INTO statement.
Here's a good quick article on it and other user-defined functions. If you still have a driving need for a stored procedure, you can wrap the inline table-valued user-defined function with a stored procedure. The stored procedure just passes parameters when it calls select * from the inline table-valued user-defined function.
So for instance, you'd have an inline table-valued user-defined function to get a list of customers for a particular region:
CREATE FUNCTION CustomersByRegion
(
#RegionID int
)
RETURNS TABLE
AS
RETURN
SELECT *
FROM customers
WHERE RegionID = #RegionID
GO
You can then call this function to get what your results a such:
SELECT * FROM CustomersbyRegion(1)
Or to do a SELECT INTO:
SELECT * INTO CustList FROM CustomersbyRegion(1)
If you still need a stored procedure, then wrap the function as such:
CREATE PROCEDURE uspCustomersByRegion
(
#regionID int
)
AS
BEGIN
SELECT * FROM CustomersbyRegion(#regionID);
END
GO
I think this is the most 'hack-less' method to obtain the desired results. It uses the existing features as they were intended to be used without additional complications. By nesting the inline table-valued user-defined function in the stored procedure, you have access to the functionality in two ways. Plus! You have only one point of maintenance for the actual SQL code.
The use of OPENROWSET has been suggested, but this is not what the OPENROWSET function was intended to be used for (From Books Online):
Includes all connection information
that is required to access remote data
from an OLE DB data source. This
method is an alternative to accessing
tables in a linked server and is a
one-time, ad hoc method of connecting
and accessing remote data by using OLE
DB. For more frequent references to
OLE DB data sources, use linked
servers instead.
Using OPENROWSET will get the job done, but it will incur some additional overhead for opening up local connections and marshalling data. It also may not be an option in all cases since it requires an ad hoc query permission which poses a security risk and therefore may not be desired. Also, the OPENROWSET approach will preclude the use of stored procedures returning more than one result set. Wrapping multiple inline table-value user-defined functions in a single stored procedure can achieve this.
Select ##ServerName
EXEC sp_serveroption ##ServerName, 'DATA ACCESS', TRUE
SELECT *
INTO #tmpTable
FROM OPENQUERY(YOURSERVERNAME, 'EXEC db.schema.sproc 1')
Easiest Solution:
CREATE TABLE #temp (...);
INSERT INTO #temp
EXEC [sproc];
If you don't know the schema then you can do the following. Please
note that there are severe security risks in this method.
SELECT *
INTO #temp
FROM OPENROWSET('SQLNCLI',
'Server=localhost;Trusted_Connection=yes;',
'EXEC [db].[schema].[sproc]')
When the stored procedure returns a lot of columns and you do not want to manually "create" a temporary table to hold the result, I've found the easiest way is to go into the stored procedure and add an "into" clause on the last select statement and add 1=0 to the where clause.
Run the stored procedure once and go back and remove the SQL code you just added. Now, you'll have an empty table matching the stored procedure's result. You could either "script table as create" for a temporary table or simply insert directly into that table.
declare #temp table
(
name varchar(255),
field varchar(255),
filename varchar(255),
filegroup varchar(255),
size varchar(255),
maxsize varchar(255),
growth varchar(255),
usage varchar(255)
);
INSERT #temp Exec sp_helpfile;
select * from #temp;
If the results table of your stored proc is too complicated to type out the "create table" statement by hand, and you can't use OPENQUERY OR OPENROWSET, you can use sp_help to generate the list of columns and data types for you. Once you have the list of columns, it's just a matter of formatting it to suit your needs.
Step 1: Add "into #temp" to the output query (e.g. "select [...] into #temp from [...]").
The easiest way is to edit the output query in the proc directly. if you can't change the stored proc, you can copy the contents into a new query window and modify the query there.
Step 2: Run sp_help on the temp table. (e.g. "exec tempdb..sp_help #temp")
After creating the temp table, run sp_help on the temp table to get a list of the columns and data types including the size of varchar fields.
Step 3: Copy the data columns & types into a create table statement
I have an Excel sheet that I use to format the output of sp_help into a "create table" statement. You don't need anything that fancy, just copy and paste into your SQL editor. Use the column names, sizes, and types to construct a "Create table #x [...]" or "declare #x table [...]" statement which you can use to INSERT the results of the stored procedure.
Step 4: Insert into the newly created table
Now you'll have a query that's like the other solutions described in this thread.
DECLARE #t TABLE
(
--these columns were copied from sp_help
COL1 INT,
COL2 INT
)
INSERT INTO #t
Exec spMyProc
This technique can also be used to convert a temp table (#temp) to a table variable (#temp). While this may be more steps than just writing the create table statement yourself, it prevents manual error such as typos and data type mismatches in large processes. Debugging a typo can take more time than writing the query in the first place.
Does your stored procedure only retrieve the data or modify it too? If it's used only for retrieving, you can convert the stored procedure into a function and use the Common Table Expressions (CTEs) without having to declare it, as follows:
with temp as (
select * from dbo.fnFunctionName(10, 20)
)
select col1, col2 from temp
However, whatever needs to be retrieved from the CTE should be used in one statement only. You cannot do a with temp as ... and try to use it after a couple of lines of SQL. You can have multiple CTEs in one statement for more complex queries.
For example,
with temp1020 as (
select id from dbo.fnFunctionName(10, 20)
),
temp2030 as (
select id from dbo.fnFunctionName(20, 30)
)
select * from temp1020
where id not in (select id from temp2030)
If the OPENROWSET is causing you issues, there is another way from 2012 onwards; make use of sys.dm_exec_describe_first_result_set_for_object, as mentioned here: Retrieve column names and types of a stored procedure?
First, create this stored procedure to generate the SQL for the temporary table:
CREATE PROCEDURE dbo.usp_GetStoredProcTableDefinition(
#ProcedureName nvarchar(128),
#TableName nvarchar(128),
#SQL nvarchar(max) OUTPUT
)
AS
SET #SQL = 'CREATE TABLE ' + #tableName + ' ('
SELECT #SQL = #SQL + '['+name +'] '+ system_type_name +'' + ','
FROM sys.dm_exec_describe_first_result_set_for_object
(
OBJECT_ID(#ProcedureName),
NULL
);
--Remove trailing comma
SET #SQL = SUBSTRING(#SQL,0,LEN(#SQL))
SET #SQL = #SQL +')'
To use the procedure, call it in the following way:
DECLARE #SQL NVARCHAR(MAX)
exec dbo.usp_GetStoredProcTableDefinition
#ProcedureName='dbo.usp_YourProcedure',
#TableName='##YourGlobalTempTable',#SQL = #SQL OUTPUT
INSERT INTO ##YourGlobalTempTable
EXEC [dbo].usp_YourProcedure
select * from ##YourGlobalTempTable
Note that I'm using a global temporary table. That's because using EXEC to run the dynamic SQL creates its own session, so an ordinary temporary table would be out of scope to any subsequent code. If a global temporary table is a problem, you can use an ordinary temporary table, but any subsequent SQL would need to be dynamic, that is, also executed by the EXEC statement.
Quassnoi put me most of the way there, but one thing was missing:
****I needed to use parameters in the stored procedure.****
And OPENQUERY does not allow for this to happen:
So I found a way to work the system and also not have to make the table definition so rigid, and redefine it inside another stored procedure (and of course take the chance it may break)!
Yes, you can dynamically create the table definition returned from the stored procedure by
using the OPENQUERY statement with bogus varaiables (as long the NO RESULT SET returns the
same number of fields and in the same position as a dataset with good data).
Once the table is created, you can use exec stored procedure into the temporary table all day long.
And to note (as indicated above) you must enable data access,
EXEC sp_serveroption 'MYSERVERNAME', 'DATA ACCESS', TRUE
Code:
declare #locCompanyId varchar(8)
declare #locDateOne datetime
declare #locDateTwo datetime
set #locDateOne = '2/11/2010'
set #locDateTwo = getdate()
--Build temporary table (based on bogus variable values)
--because we just want the table definition and
--since openquery does not allow variable definitions...
--I am going to use bogus variables to get the table defintion.
select * into #tempCoAttendanceRpt20100211
FROM OPENQUERY(DBASESERVER,
'EXEC DATABASE.dbo.Proc_MyStoredProc 1,"2/1/2010","2/15/2010 3:00 pm"')
set #locCompanyId = '7753231'
insert into #tempCoAttendanceRpt20100211
EXEC DATABASE.dbo.Proc_MyStoredProc #locCompanyId,#locDateOne,#locDateTwo
set #locCompanyId = '9872231'
insert into #tempCoAttendanceRpt20100211
EXEC DATABASE.dbo.Proc_MyStoredProc #locCompanyId,#locDateOne,#locDateTwo
select * from #tempCoAttendanceRpt20100211
drop table #tempCoAttendanceRpt20100211
Thanks for the information which was provided originally...
Yes, finally I do not have to create all these bogus (strict) table defintions when using data from
another stored procedure or database, and yes you can use parameters too.
Search reference tags:
SQL 2005 stored procedure into temp table
openquery with stored procedure and variables 2005
openquery with variables
execute stored procedure into temp table
Update: this will not work with temporary tables so I had to resort to manually creating the temporary table.
Bummer notice: this will not work with temporary tables, http://www.sommarskog.se/share_data.html#OPENQUERY
Reference: The next thing is to define LOCALSERVER. It may look like a keyword in the example, but it is in fact only a name. This is how you do it:
sp_addlinkedserver #server = 'LOCALSERVER', #srvproduct = '',
#provider = 'SQLOLEDB', #datasrc = ##servername
To create a linked server, you must have the permission ALTER ANY SERVER, or be a member of any of the fixed server roles sysadmin or setupadmin.
OPENQUERY opens a new connection to SQL Server. This has some implications:
The procedure that you call with OPENQUERY cannot refer temporary tables created in the current connection.
The new connection has its own default database (defined with sp_addlinkedserver, default is master), so all object specification must include a database name.
If you have an open transaction and are holding locks when you call OPENQUERY, the called procedure can not access what you lock. That is, if you are not careful you will block yourself.
Connecting is not for free, so there is a performance penalty.
If you're lucky enough to have SQL 2012 or higher, you can use dm_exec_describe_first_result_set_for_object
I have just edited the sql provided by gotqn. Thanks gotqn.
This creates a global temp table with name same as procedure name. The temp table can later be used as required. Just don't forget to drop it before re-executing.
declare #procname nvarchar(255) = 'myProcedure',
#sql nvarchar(max)
set #sql = 'create table ##' + #procname + ' ('
begin
select #sql = #sql + '[' + r.name + '] ' + r.system_type_name + ','
from sys.procedures AS p
cross apply sys.dm_exec_describe_first_result_set_for_object(p.object_id, 0) AS r
where p.name = #procname
set #sql = substring(#sql,1,len(#sql)-1) + ')'
execute (#sql)
execute('insert ##' + #procname + ' exec ' + #procname)
end
This stored proc does the job:
CREATE PROCEDURE [dbo].[ExecIntoTable]
(
#tableName NVARCHAR(256),
#storedProcWithParameters NVARCHAR(MAX)
)
AS
BEGIN
DECLARE #driver VARCHAR(10)
DECLARE #connectionString NVARCHAR(600)
DECLARE #sql NVARCHAR(MAX)
DECLARE #rowsetSql NVARCHAR(MAX)
SET #driver = '''SQLNCLI'''
SET #connectionString =
'''server=' +
CAST(SERVERPROPERTY('ServerName') AS NVARCHAR(256)) +
COALESCE('\' + CAST(SERVERPROPERTY('InstanceName') AS NVARCHAR(256)), '') +
';trusted_connection=yes'''
SET #rowsetSql = '''EXEC ' + REPLACE(#storedProcWithParameters, '''', '''''') + ''''
SET #sql = '
SELECT
*
INTO
' + #tableName + '
FROM
OPENROWSET(' + #driver + ',' + #connectionString + ',' + #rowsetSql + ')'
EXEC (#sql)
END
GO
It's a slight rework of this: Insert stored procedure results into table so that it actually works.
If you want it to work with a temporary table then you will need to use a ##GLOBAL table and drop it afterwards.
In order to insert the first record set of a stored procedure into a temporary table you need to know the following:
only the first row set of the stored procedure can be inserted into a temporary table
the stored procedure must not execute dynamic T-SQL statement (sp_executesql)
you need to define the structure of the temporary table first
The above may look as limitation, but IMHO it perfectly makes sense - if you are using sp_executesql you can once return two columns and once ten, and if you have multiple result sets, you cannot insert them into several tables as well - you can insert maximum in two table in one T-SQL statement (using OUTPUT clause and no triggers).
So, the issue is mainly how to define the temporary table structure before performing the EXEC ... INTO ... statement.
sys.dm_exec_describe_first_result_set_for_object
sys.dm_exec_describe_first_result_set
sp_describe_first_result_set
The first works with OBJECT_ID while the second and the third works with Ad-hoc queries as well. I prefer to use the DMV instead of the sp as you can use CROSS APPLY and build the temporary table definitions for multiple procedures at the same time.
SELECT p.name, r.*
FROM sys.procedures AS p
CROSS APPLY sys.dm_exec_describe_first_result_set_for_object(p.object_id, 0) AS r;
Also, pay attention to the system_type_name field as it can be very useful. It stores the column complete definition. For, example:
smalldatetime
nvarchar(max)
uniqueidentifier
nvarchar(1000)
real
smalldatetime
decimal(18,2)
and you can use it directly in most of the cases to create the table definition.
So, I think in most of the cases (if the stored procedure match certain criteria) you can easily build dynamic statements for solving such issues (create the temporary table, insert the stored procedure result in it, do what you need with the data).
Note, that the objects above fail to define the first result set data in some cases like when dynamic T-SQL statements are executed or temporary tables are used in the stored procedure.
I'm creating a table with the following schema and data.
Create a stored procedure.
Now I know what the result of my procedure is, so I am performing the following query.
CREATE TABLE [dbo].[tblTestingTree](
[Id] [int] IDENTITY(1,1) NOT NULL,
[ParentId] [int] NULL,
[IsLeft] [bit] NULL,
[IsRight] [bit] NULL,
CONSTRAINT [PK_tblTestingTree] PRIMARY KEY CLUSTERED
(
[Id] ASC
) WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
SET IDENTITY_INSERT [dbo].[tblTestingTree] ON
INSERT [dbo].[tblTestingTree] ([Id], [ParentId], [IsLeft], [IsRight]) VALUES (1, NULL, NULL, NULL)
INSERT [dbo].[tblTestingTree] ([Id], [ParentId], [IsLeft], [IsRight]) VALUES (2, 1, 1, NULL)
INSERT [dbo].[tblTestingTree] ([Id], [ParentId], [IsLeft], [IsRight]) VALUES (3, 1, NULL, 1)
INSERT [dbo].[tblTestingTree] ([Id], [ParentId], [IsLeft], [IsRight]) VALUES (4, 2, 1, NULL)
INSERT [dbo].[tblTestingTree] ([Id], [ParentId], [IsLeft], [IsRight]) VALUES (5, 2, NULL, 1)
INSERT [dbo].[tblTestingTree] ([Id], [ParentId], [IsLeft], [IsRight]) VALUES (6, 3, 1, NULL)
INSERT [dbo].[tblTestingTree] ([Id], [ParentId], [IsLeft], [IsRight]) VALUES (7, 3, NULL, 1)
INSERT [dbo].[tblTestingTree] ([Id], [ParentId], [IsLeft], [IsRight]) VALUES (8, 4, 1, NULL)
INSERT [dbo].[tblTestingTree] ([Id], [ParentId], [IsLeft], [IsRight]) VALUES (9, 4, NULL, 1)
INSERT [dbo].[tblTestingTree] ([Id], [ParentId], [IsLeft], [IsRight]) VALUES (10, 5, 1, NULL)
SET IDENTITY_INSERT [dbo].[tblTestingTree] OFF
VALUES (10, 5, 1, NULL)
SET IDENTITY_INSERT [dbo].[tblTestingTree] On
create procedure GetDate
as
begin
select Id,ParentId from tblTestingTree
end
create table tbltemp
(
id int,
ParentId int
)
insert into tbltemp
exec GetDate
select * from tbltemp;
If the query doesn't contain parameter, use OpenQuery else use OpenRowset.
Basic thing would be to create schema as per stored procedure and insert into that table. e.g.:
DECLARE #abc TABLE(
RequisitionTypeSourceTypeID INT
, RequisitionTypeID INT
, RequisitionSourcingTypeID INT
, AutoDistOverride INT
, AllowManagerToWithdrawDistributedReq INT
, ResumeRequired INT
, WarnSupplierOnDNRReqSubmission INT
, MSPApprovalReqd INT
, EnableMSPSupplierCounterOffer INT
, RequireVendorToAcceptOffer INT
, UseCertification INT
, UseCompetency INT
, RequireRequisitionTemplate INT
, CreatedByID INT
, CreatedDate DATE
, ModifiedByID INT
, ModifiedDate DATE
, UseCandidateScheduledHours INT
, WeekEndingDayOfWeekID INT
, AllowAutoEnroll INT
)
INSERT INTO #abc
EXEC [dbo].[usp_MySp] 726,3
SELECT * FROM #abc
Code
CREATE TABLE #T1
(
col1 INT NOT NULL,
col2 NCHAR(50) NOT NULL,
col3 TEXT NOT NULL,
col4 DATETIME NULL,
col5 NCHAR(50) NULL,
col6 CHAR(2) NULL,
col6 NCHAR(100) NULL,
col7 INT NULL,
col8 NCHAR(50) NULL,
col9 DATETIME NULL,
col10 DATETIME NULL
)
DECLARE #Para1 int
DECLARE #Para2 varchar(32)
DECLARE #Para3 varchar(100)
DECLARE #Para4 varchar(15)
DECLARE #Para5 varchar (12)
DECLARE #Para6 varchar(1)
DECLARE #Para7 varchar(1)
SET #Para1 = 1025
SET #Para2 = N'6as54fsd56f46sd4f65sd'
SET #Para3 = N'XXXX\UserName'
SET #Para4 = N'127.0.0.1'
SET #Para5 = N'XXXXXXX'
SET #Para6 = N'X'
SET #Para7 = N'X'
INSERT INTO #T1
(
col1,
col2,
col3,
col4,
col5,
col6,
col6,
col7,
col8,
col9,
col10,
)
EXEC [dbo].[usp_ProcedureName] #Para1, #Para2, #Para3, #Para4, #Para5, #Para6, #Para6
I hope this helps. Please qualify as appropriate.
I found Passing Arrays/DataTables into Stored Procedures which might give you another idea on how you might go solving your problem.
The link suggests to use an Image type parameter to pass into the stored procedure. Then in the stored procedure, the image is transformed into a table variable containing the original data.
Maybe there is a way this can be used with a temporary table.
I met the same problem and here is what I did for this from Paul's suggestion. The main part is here is to use NEWID() to avoid multiple users run the store procedures/scripts at the same time, the pain for global temporary table.
DECLARE #sql varchar(max) = '',
#tmp_global_table varchar(255) = '##global_tmp_' + CONVERT(varchar(36), NEWID())
SET #sql = #sql + 'select * into [' + #tmp_global_table + '] from YOURTABLE'
EXEC(#sql)
EXEC('SELECT * FROM [' + #tmp_global_table + ']')
Another method is to create a type and use PIPELINED to then pass back your object. This is limited to knowing the columns however. But it has the advantage of being able to do:
SELECT *
FROM TABLE(CAST(f$my_functions('8028767') AS my_tab_type))
This can be done in SQL Server 2014+ provided the stored procedure only returns one table. If anyone finds a way of doing this for multiple tables I'd love to know about it.
DECLARE #storedProcname NVARCHAR(MAX) = ''
SET #storedProcname = 'myStoredProc'
DECLARE #strSQL AS VARCHAR(MAX) = 'CREATE TABLE myTableName '
SELECT #strSQL = #strSQL+STUFF((
SELECT ',' +name+' ' + system_type_name
FROM sys.dm_exec_describe_first_result_set_for_object (OBJECT_ID(#storedProcname),0)
FOR XML PATH('')
),1,1,'(') + ')'
EXEC (#strSQL)
INSERT INTO myTableName
EXEC ('myStoredProc #param1=1, #param2=2')
SELECT * FROM myTableName
DROP TABLE myTableName
This pulls the definition of the returned table from system tables, and uses that to build the temp table for you. You can then populate it from the stored procedure as stated before.
There are also variants of this that work with Dynamic SQL too.
After searching around I found a way to create a temp table dynamically for any stored procedure without using OPENROWSET or OPENQUERY using a generic schema of Stored Procedure's result definition especially when you are not database Administrator.
Sql server has a buit-in proc sp_describe_first_result_set that can provide you with schema of any procedures resultset. I created a schema table from results of this procedure and manually set all the field to NULLABLE.
declare #procname varchar(100) = 'PROCEDURENAME' -- your procedure name
declare #param varchar(max) = '''2019-06-06''' -- your parameters
declare #execstr nvarchar(max) = N'exec ' + #procname
declare #qry nvarchar(max)
-- Schema table to store the result from sp_describe_first_result_set.
create table #d
(is_hidden bit NULL, column_ordinal int NULL, name sysname NULL, is_nullable bit NULL, system_type_id int NULL, system_type_name nvarchar(256) NULL,
max_length smallint NULL, precision tinyint NULL, scale tinyint NULL, collation_name sysname NULL, user_type_id int NULL, user_type_database sysname NULL,
user_type_schema sysname NULL,user_type_name sysname NULL,assembly_qualified_type_name nvarchar(4000),xml_collection_id int NULL,xml_collection_database sysname NULL,
xml_collection_schema sysname NULL,xml_collection_name sysname NULL,is_xml_document bit NULL,is_case_sensitive bit NULL,is_fixed_length_clr_type bit NULL,
source_server sysname NULL,source_database sysname NULL,source_schema sysname NULL,source_table sysname NULL,source_column sysname NULL,is_identity_column bit NULL,
is_part_of_unique_key bit NULL,is_updateable bit NULL,is_computed_column bit NULL,is_sparse_column_set bit NULL,ordinal_in_order_by_list smallint NULL,
order_by_list_length smallint NULL,order_by_is_descending smallint NULL,tds_type_id int NULL,tds_length int NULL,tds_collation_id int NULL,
tds_collation_sort_id tinyint NULL)
-- Get result set definition of your procedure
insert into #d
EXEC sp_describe_first_result_set #exestr, NULL, 0
-- Create a query to generate and populate a global temp table from above results
select
#qry = 'Create table ##t(' +
stuff(
(select ',' + name + ' '+ system_type_name + ' NULL'
from #d d For XML Path, TYPE)
.value(N'.[1]', N'nvarchar(max)')
, 1,1,'')
+ ')
insert into ##t
Exec '+#procname+' ' + #param
Exec sp_executesql #qry
-- Use below global temp table to query the data as you may
select * from ##t
-- **WARNING** Don't forget to drop the global temp table ##t.
--drop table ##t
drop table #d
Developed and tested on Sql Server version - Microsoft SQL Server 2016 (RTM) - 13.0.1601.5(Build 17134:)
You can tweak the schema for your SQL server version that you are using (if needed).
It's a simple 2 step process:
- create a temporary table
- Insert into the temporary table.
Code to perform the same:
CREATE TABLE #tempTable (Column1 int, Column2 varchar(max));
INSERT INTO #tempTable
EXEC [app].[Sproc_name]
#param1 = 1,
#param2 =2;
If you know the parameters that are being passed and if you don't have access to make sp_configure, then edit the stored procedure with these parameters and the same can be stored in a ##global table.
A few years late to the question, but I needed something like this for some quick and dirty code generation. I believe as others have stated it is just easier to define the temp table up front, but this method should work for simple stored procedure queries or sql statments.
This will be a little convoluted, but it borrows from the contributors here as well as Paul White's solution from DBA Stack Exchange Get stored procedure result column-types. Again, to reiterate this approach & example is not designed for processes in a multi user environment. In this case the table definition is being set for a short time in a global temp table for reference by a code generation template process.
I haven't fully tested this so there may be caveats so you may want to go to the MSDN link in Paul White's answer. This applies to SQL 2012 and higher.
First use the stored procedure sp_describe_first_result_set which resembles Oracle's describe.
This will evaluate the first row of the first result set so if your stored procedure or statement returns multiple queries it will only describe the first result.
I created a stored proc to break down the tasks that returns a single field to select from to create the temp table definition.
CREATE OR ALTER PROCEDURE [dbo].[sp_GetTableDefinitionFromSqlBatch_DescribeFirstResultSet]
(
#sql NVARCHAR(4000)
,#table_name VARCHAR(100)
,#TableDefinition NVARCHAR(MAX) OUTPUT
)
AS
BEGIN
SET NOCOUNT ON
DECLARE #TempTableDefinition NVARCHAR(MAX)
DECLARE #NewLine NVARCHAR(4) = CHAR(13)+CHAR(10)
DECLARE #ResultDefinition TABLE ( --The View Definition per MSDN
is_hidden bit NOT NULL
, column_ordinal int NOT NULL
, [name] sysname NULL
, is_nullable bit NOT NULL
, system_type_id int NOT NULL
, system_type_name nvarchar(256) NULL
, max_length smallint NOT NULL
, [precision] tinyint NOT NULL
, scale tinyint NOT NULL
, collation_name sysname NULL
, user_type_id int NULL
, user_type_database sysname NULL
, user_type_schema sysname NULL
, user_type_name sysname NULL
, assembly_qualified_type_name nvarchar(4000)
, xml_collection_id int NULL
, xml_collection_database sysname NULL
, xml_collection_schema sysname NULL
, xml_collection_name sysname NULL
, is_xml_document bit NOT NULL
, is_case_sensitive bit NOT NULL
, is_fixed_length_clr_type bit NOT NULL
, source_server sysname NULL
, source_database sysname NULL
, source_schema sysname NULL
, source_table sysname NULL
, source_column sysname NULL
, is_identity_column bit NULL
, is_part_of_unique_key bit NULL
, is_updateable bit NULL
, is_computed_column bit NULL
, is_sparse_column_set bit NULL
, ordinal_in_order_by_list smallint NULL
, order_by_is_descending smallint NULL
, order_by_list_length smallint NULL
, tds_type_id int NOT NULL
, tds_length int NOT NULL
, tds_collation_id int NULL
, tds_collation_sort_id tinyint NULL
)
--Insert the description into table variable
INSERT #ResultDefinition
EXEC sp_describe_first_result_set #sql
--Now Build the string to create the table via union select statement
;WITH STMT AS (
SELECT N'CREATE TABLE ' + #table_name + N' (' AS TextVal
UNION ALL
SELECT
CONCAT(
CASE column_ordinal
WHEN 1 THEN ' ' ELSE ' , ' END --Determines if comma should precede
, QUOTENAME([name]) , ' ', system_type_name -- Column Name and SQL TYPE
,CASE is_nullable
WHEN 0 THEN ' NOT NULL' ELSE ' NULL' END --NULLABLE CONSTRAINT
) AS TextVal
FROM #ResultDefinition WHERE is_hidden = 0 -- May not be needed
UNION ALL
SELECT N');' + #NewLine
)
--Now Combine the rows to a single String
SELECT #TempTableDefinition = COALESCE (#TempTableDefinition + #NewLine + TextVal, TextVal) FROM STMT
SELECT #TableDefinition = #TempTableDefinition
END
The conundrum is that you need to use a global table, but you need to make it unique enough
so you can drop and create from it frequently without worrying about a collision.
In the example I used a Guid (FE264BF5_9C32_438F_8462_8A5DC8DEE49E) for the global variable replacing the hyphens with underscore
DECLARE #sql NVARCHAR(4000) = N'SELECT ##SERVERNAME as ServerName, GETDATE() AS Today;'
DECLARE #GlobalTempTable VARCHAR(100) = N'##FE264BF5_9C32_438F_8462_8A5DC8DEE49E_MyTempTable'
--#sql can be a stored procedure name like dbo.foo without parameters
DECLARE #TableDef NVARCHAR(MAX)
DROP TABLE IF EXISTS #MyTempTable
DROP TABLE IF EXISTS ##FE264BF5_9C32_438F_8462_8A5DC8DEE49E_MyTempTable
EXEC [dbo].[sp_GetTableDefinitionFromSqlBatch_DescribeFirstResultSet]
#sql, #GlobalTempTable, #TableDef OUTPUT
--Creates the global table ##FE264BF5_9C32_438F_8462_8A5DC8DEE49E_MyTempTable
EXEC sp_executesql #TableDef
--Now Call the stored procedure, SQL Statement with Params etc.
INSERT ##FE264BF5_9C32_438F_8462_8A5DC8DEE49E_MyTempTable
EXEC sp_executesql #sql
--Select the results into your undefined Temp Table from the Global Table
SELECT *
INTO #MyTempTable
FROM ##FE264BF5_9C32_438F_8462_8A5DC8DEE49E_MyTempTable
SELECT * FROM #MyTempTable
DROP TABLE IF EXISTS #MyTempTable
DROP TABLE IF EXISTS ##FE264BF5_9C32_438F_8462_8A5DC8DEE49E_MyTempTable
Again, I have only tested it with simple stored procedure queries and simple queries so your mileage may vary. Hope this helps someone.
Here is my T-SQL with parameters
--require one time execution if not configured before
sp_configure 'Show Advanced Options', 1
GO
RECONFIGURE
GO
--require one time execution if not configured before
sp_configure 'Ad Hoc Distributed Queries', 1
GO
RECONFIGURE
GO
--the query
DECLARE #param1 int = 1, #param2 int = 2
DECLARE #SQLStr varchar(max) = 'SELECT * INTO #MyTempTable
FROM OPENROWSET(''SQLNCLI'',
''Server=ServerName;Database=DbName;Trusted_Connection=yes'',
''exec StoredProcedureName '+ CAST(#param1 AS varchar(15)) +','+ CAST(#param2 AS varchar(15)) +''') AS a ;
select * from #MyTempTable;
drop table #MyTempTable
';
EXECUTE(#SQLStr);
Well, you do have to create a temp table, but it doesn't have to have the right schema....I've created a stored procedure that modifies an existing temp table so that it has the required columns with the right data type and order (dropping all existing columns, adding new columns):
GO
create procedure #TempTableForSP(#tableId int, #procedureId int)
as
begin
declare #tableName varchar(max) = (select name
from tempdb.sys.tables
where object_id = #tableId
);
declare #tsql nvarchar(max);
declare #tempId nvarchar(max) = newid();
set #tsql = '
declare #drop nvarchar(max) = (select ''alter table tempdb.dbo.' + #tableName
+ ' drop column '' + quotename(c.name) + '';''+ char(10)
from tempdb.sys.columns c
where c.object_id = ' +
cast(#tableId as varchar(max)) + '
for xml path('''')
)
alter table tempdb.dbo.' + #tableName + ' add ' + QUOTENAME(#tempId) + ' int;
exec sp_executeSQL #drop;
declare #add nvarchar(max) = (
select ''alter table ' + #tableName
+ ' add '' + name
+ '' '' + system_type_name
+ case when d.is_nullable=1 then '' null '' else '''' end
+ char(10)
from sys.dm_exec_describe_first_result_set_for_object('
+ cast(#procedureId as varchar(max)) + ', 0) d
order by column_ordinal
for xml path(''''))
execute sp_executeSQL #add;
alter table ' + #tableName + ' drop column ' + quotename(#tempId) + ' ';
execute sp_executeSQL #tsql;
end
GO
create table #exampleTable (pk int);
declare #tableId int = object_Id('tempdb..#exampleTable')
declare #procedureId int = object_id('examplestoredProcedure')
exec #TempTableForSP #tableId, #procedureId;
insert into #exampleTable
exec examplestoredProcedure
Note this won't work if sys.dm_exec_describe_first_result_set_for_object can't determine the results of the stored procedure (for instance if it uses a temp table).
If you let dynamic SQL create a temp table, this table is owned by the Dynamic SQL connection, as opposed to the connection your stored procedure is called from.
DECLARE #COMMA_SEPARATED_KEYS varchar(MAX);
DROP TABLE IF EXISTS KV;
CREATE TABLE KV (id_person int, mykey varchar(30), myvalue int);
INSERT INTO KV VALUES
(1, 'age', 16),
(1, 'weight', 63),
(1, 'height', 175),
(2, 'age', 26),
(2, 'weight', 83),
(2, 'height', 185);
WITH cte(mykey) AS (
SELECT DISTINCT mykey FROM KV
)
SELECT #COMMA_SEPARATED_KEYS=STRING_AGG(mykey,',') FROM cte;
SELECT #COMMA_SEPARATED_KEYS AS keys;
DECLARE #ExecuteExpression varchar(MAX);
DROP TABLE IF EXISTS #Pivoted;
SET #ExecuteExpression = N'
SELECT *
INTO #Pivoted
FROM
(
SELECT
mykey,
myvalue,
id_person
FROM KV
) AS t
PIVOT(
MAX(t.myvalue)
FOR mykey IN (COMMA_SEPARATED_KEYS)
) AS pivot_table;
';
SET #ExecuteExpression = REPLACE(#ExecuteExpression, 'COMMA_SEPARATED_KEYS', #COMMA_SEPARATED_KEYS);
EXEC(#ExecuteExpression);
SELECT * FROM #Pivoted;
Msg 208, Level 16, State 0
Invalid object name '#Pivoted'.
This is because #Pivoted is owned by the Dynamic SQL connection. So the last instruction
SELECT * FROM #Pivoted
fails.
One way to not face this issue is to make sure all references to #Pivoted are made from inside the dynamic query itself:
DECLARE #COMMA_SEPARATED_KEYS varchar(MAX);
DROP TABLE IF EXISTS KV;
CREATE TABLE KV (id_person int, mykey varchar(30), myvalue int);
INSERT INTO KV VALUES
(1, 'age', 16),
(1, 'weight', 63),
(1, 'height', 175),
(2, 'age', 26),
(2, 'weight', 83),
(2, 'height', 185);
WITH cte(mykey) AS (
SELECT DISTINCT mykey FROM KV
)
SELECT #COMMA_SEPARATED_KEYS=STRING_AGG(mykey,',') FROM cte;
SELECT #COMMA_SEPARATED_KEYS AS keys;
DECLARE #ExecuteExpression varchar(MAX);
DROP TABLE IF EXISTS #Pivoted;
SET #ExecuteExpression = N'
SELECT *
INTO #Pivoted
FROM
(
SELECT
mykey,
myvalue,
id_person
FROM KV
) AS t
PIVOT(
MAX(t.myvalue)
FOR mykey IN (COMMA_SEPARATED_KEYS)
) AS pivot_table;
SELECT * FROM #Pivoted;
';
SET #ExecuteExpression = REPLACE(#ExecuteExpression, 'COMMA_SEPARATED_KEYS', #COMMA_SEPARATED_KEYS);
EXEC(#ExecuteExpression);
First, modify your stored procedure to save the end results in to a temp table. By doing this we are creating a table matching with the SP output fields. And then have a select statement to save that temp table to a any table name. Then execute the SP as explained in step 2
Step 1: modify your stored procedure to save the end results in to a temp table
[your stored procedure]
into #table_temp //this will insert the data to a temp table
from #table_temp
select * into SP_Output_Table_1 from #table_temp //this will save data to a actual table
Step 2: Execute the SP as below that will insert records to your table
Insert SP_Output_Table_1
EXE You_SP_Nane #Parameter1 = 52, #parameter2 =1

Show a few columns in a stored procedure based on a parameter

I have a parameter in my stored procedure called internal. If "internal" = yes then I want to display an additional 2 columns in my results. If it's no I don't want to display these columns.
I can do a case statement and then I can set the column to be empty but the column name will still be returned in the results.
My questions are:
Is there a way not to return these 2 columns in the results at all?
Can I do it in one case statement and not a separate case statement for each column?
Thank you
No, CASE is a function, and can only return a single value.
And According to your comment:-
The issue with 2 select statements are that it's a major complicated
select statement and I really don't want to have to have the whole
select statement twice.
so you can use the next approach for avoid duplicate code:-
Create procedure proc_name (#internal char(3), .... others)
as
BEGIN
declare #AddationalColumns varchar(100)
set #AddationalColumns = ''
if #internal = 'Yes'
set #AddationalColumns = ',addtionalCol1 , addtionalCol2'
exec ('Select
col1,
col2,
col3'
+ #AddationalColumns +
'From
tableName
Where ....
' )
END
Try IF Condition
IF(internal = 'yes')
BEGIN
SELECT (Columns) FROM Table1
END
ELSE
BEGIN
SELECT (Columns With additional 2 columns) FROM Table1
END
You can do something like this solution. It allows you to keep only one copy of code if it's so important but you will have to deal with dynamic SQL.
CREATE TABLE tab (col1 INT, col2 INT, col3 INT);
GO
DECLARE #internal BIT = 0, #sql NVARCHAR(MAX)
SET #sql = N'SELECT col1 ' + (SELECT CASE #internal WHEN 1 THEN N', col2, col3 ' ELSE N'' END) + N'FROM tab'
EXEC sp_executesql #sql
GO
DROP TABLE tab
GO
Another option is to create a 'wrapper' proc. Keep your current one untouched.
Create a new proc which executes this (pseudo code):
create proc schema.wrapperprocname (same #params as current proc)
as
begin
Create table #output (column list & datatypes from current proc);
insert into #output
exec schema.currentproc (#params)
if #internal = 'Yes'
select * from #output
else
select columnlist without the extra 2 columns from #output
end
This way the complex select statement remains encapsulated in the original proc.
Your only overhead is keeping the #output table & select lists in in this proc in sync with any changes to the original proc.
IMO it's also easier to understand/debug/tune than dynamic sql.

Split A SELECT query into several Xml files

This is a simplified version of my problem.
Using the following SELECT statement I output my data as Xml.
SET #result = (
SELECT * FROM MyTable
FOR XML path('receipts')
)
SELECT #result AS xmloutput
I then write the output into an Xml file that uses a timestamp in the filename:
SET #sqlCommand =
'bcp "EXEC ' +#db + '.dbo.MyStoreProcedure" queryout "' + #filePath + #fileName + ' " -T -C 1252 -w'
EXEC master..xp_cmdshell #sqlCommand
In the snippet above:
MyStoreProcedure is basically the code from the first snippet i.e. the SELECT results.
#filename has a structure like YYYYMMDDHHMMSS_customers.xml.
Now the problem is, the program that is supposed to read this Xml file has a limit of 20000 records. What I should do is split the results into separate Xml files. So if the original SELECT query gives 25000 records. They should be split into two non-overlapping files: First one containing 20000 and the second the remaining 5000.
To avoid overwriting on the same file (in case the process goes very fast), we can have a 1-second wait between each batch so that the next file will get a new name.
How can I implement this split?
Thanks in advance.
The function NTILE (Starting with SQL Server 2012) allows you to group your data. If you have a lower version you can easily simulate this with a combination of ROW_NUMBER and integer division.
The following writes your table rows together with a chunk number into a temp table. The WHILE takes chunk after chunk. You can use the chunk-number to add it to your file name.
The function WAITFOR (again 2012+) allows to pause automatically.
DECLARE #ApproxRowsPerChunk INT=10; --will float a little...
SELECT NTILE((SELECT Count(*) FROM sys.objects) / #ApproxRowsPerChunk) OVER(ORDER BY o.object_id) AS ChunkNumber
,*
INTO #StagingTable
FROM sys.objects AS o;
DECLARE #nr INT=1;
DECLARE #maxNr INT=(SELECT MAX(ChunkNumber) FROM #StagingTable);
WHILE #nr<=#maxNr
BEGIN
SELECT * FROM #StagingTable WHERE ChunkNumber=#nr FOR XML PATH('test');
SET #nr=#nr+1;
WAITFOR DELAY '00:00:01';
END
Hint
This would allow you, to integrate something like 1 of 17 into your XML (and into the file's name if needed).
please see if the below helps. I basically take your finished myTableresults, assign row_number() and divide them into partitions, I then loop through the partitions as per your batch size, executing your bulk copy in the loop. You may have some more work to do on dynamically setting the file names (you could use the #current_partition variable in your file name builder).
I have commented my code:
-- Dummy data, represents your results set
IF OBJECT_ID('tempdb..#t1') IS NOT NULL
DROP TABLE #t1
CREATE TABLE #t1 (
initials VARCHAR(10)
,no_cars VARCHAR(10)
)
INSERT INTO #t1
VALUES
('AA',1)
,('BB',1)
,('CC',1)
,('DD',1)
,('EE',1)
,('FF',1)
,('GG',1)
,('HH',1)
,('II',1)
,('JJ',1)
,('KK',1)
---- end of test data creation
-- Assign query partition size, in your case would be 20,000. Must be float (or other decimal).
DECLARE #partition_size FLOAT = 3;
IF OBJECT_ID('tempdb..#t2') IS NOT NULL
DROP TABLE #t2;
-- Assign your results set a row number
WITH [cte1] AS (
SELECT
initials
,no_cars
,ROW_NUMBER() OVER (ORDER BY no_cars ASC) AS row_no
FROM #t1
)
-- Assign the query partition by running a ceiling command on the row number, store the results in a temp table
SELECT
initials
,no_cars
,CEILING(row_no/#partition_size) AS query_partition
INTO #t2
FROM cte1
--- Now, create a loop to go through each partition
-- Your business variables
DECLARE #result XML
DECLARE #sqlcommand NVARCHAR(4000)
DECLARE #db VARCHAR(50) = 'db'
DECLARE #filepath VARCHAR(50) = 'C:\temp'
DECLARE #filename VARCHAR(50) = 'dynamic2017010.xml'
-- Find highest partition
DECLARE #current_partition INT = 1
DECLARE #max_partition INT = (SELECT MAX(query_partition) FROM #t2)
WHILE #current_partition <= #max_partition
BEGIN
SET #result = (
SELECT initials
,no_cars FROM #t2
WHERE query_partition = #current_partition
FOR XML path('receipts')
)
SELECT #result AS xmloutput
-- other code..?
SET #sqlCommand =
'bcp "EXEC ' +#db + '.dbo.MyStoreProcedure" queryout "' + #filePath + #fileName + ' " -T -C 1252 -w'
EXEC master..xp_cmdshell #sqlCommand
SET #current_partition += 1
END

Tempdb Full When Querying Distinct Count Of All Tables

ORIGINAL PROBLEM
I have created a custom script to retrieve data from a remote SQL server into our local copy in our office. I had some issues with the script where selected tables had some data inserted twice, thus creating duplicates. I know that for all the tables in all databases there should be no duplicates.
This issue has made me paranoid that other tables may have had this problem historically, and therefore I'd like to verify this.
SOLUTION
I have created a SQL script to insert the count and distinct count of all columns into a table for all the databases on our server (excluding the 4 system databases):
DECLARE #TableFullName AS NVARCHAR(MAX)
DECLARE #SQLQuery AS NVARCHAR(MAX)
DECLARE #TableHasDuplicates AS BIT
DECLARE #TempTableRowCount AS INT
DECLARE #ResultsTable TABLE ([CompleteTableName] NVARCHAR(200), [CountAll] INT, [CountDistinct] INT)
DECLARE #CountAll INT
DECLARE #CountDistinct INT
SET NOCOUNT ON
DECLARE #AllTables TABLE ([CompleteTableName] NVARCHAR(200))
INSERT INTO #AllTables ([CompleteTableName])
EXEC sp_msforeachdb 'SELECT ''['' + [TABLE_CATALOG] + ''].['' + [TABLE_SCHEMA] + ''].['' + [TABLE_NAME] + '']'' FROM [?].INFORMATION_SCHEMA.TABLES'
SET NOCOUNT OFF;
DECLARE [table_cursor] CURSOR FOR
(SELECT *
FROM #AllTables
WHERE [CompleteTableName] NOT LIKE '%master%' AND [CompleteTableName] NOT LIKE '%msdb%' AND [CompleteTableName] NOT LIKE '%tempdb%' AND [CompleteTableName] NOT LIKE '%model%');
OPEN [table_cursor]
PRINT N'There were ' + CAST(#CountAll AS NVARCHAR(10)) + ' tables with potential duplicate data'
FETCH NEXT FROM [table_cursor]
INTO #TableFullName
WHILE ##FETCH_STATUS = 0
BEGIN
SET #SQLQuery = 'SELECT #CntAll = COUNT(*) FROM ' + #TableFullName + ' SELECT #CntDistinct = COUNT(*) FROM (SELECT DISTINCT * FROM ' + #TableFullName + ') AS [sq] IF #CntAll > #CntDistinct SELECT #BitResult=1 ELSE SELECT #BitResult=0';
EXEC sp_executesql #SQLQuery, N'#BitResult BIT OUTPUT, #CntAll INT OUTPUT, #CntDistinct INT OUTPUT', #BitResult = #TableHasDuplicates OUTPUT, #CntAll = #CountAll OUTPUT, #CntDistinct = #CountDistinct OUTPUT;
IF #TableHasDuplicates = 1
BEGIN
INSERT INTO #ResultsTable ([CompleteTableName], [CountAll], [CountDistinct])
SELECT #TableFullName, #CountAll, #CountDistinct
END;
FETCH NEXT FROM [table_cursor]
INTO #TableFullName
END
CLOSE [table_cursor];
DEALLOCATE [table_cursor];
SELECT *
FROM #ResultsTable
An overview of how it works is the table variable #AllTables uses sp_msforeachdb with INFORMATION_SCHEMA.TABLES to list all the tables in all databases (there are 16537 tables). A table cursor is used to store all non-system entries and then I use dynamic SQL to undertake a count and distinct count which is stored in another table variable #ResultsTable.
THE PROBLEM WITH THIS SOLUTION
When I run this query, it will run for circa 3 minutes then throw an error saying that the tempdb PRIMARY filegroup is full:
I am my own DBA, and I used Brent Ozar's guide to setting up my SQL server instance, and my tempdb is set up with 8 x 3GB mdf/ndf files (the server has 8 cores):
These files show as having 23997MB available under 'General' properties.
MY QUESTIONS
If I have circa 24GB of tempdb free space, why is this relatively simple query running out of tempdb space?
Is there a better/more efficiency way of getting a count and distinct count of all tables in all databases?
You should always consider contention before adding TempDb file. Adding 7 additional TempDb file won't really help.
If I have circa 24GB of tempdb free space, why is this relatively
simple query running out of tempdb space?
No, it should not. But are you sure that you aren't dealing with large amount of data or you don't have other process running on SQL? Cursors, Temp tables and even table variables use TempDb extensively. Check which object is consuming more TempDb space:
SELECT
SUM (user_object_reserved_page_count)*8 as usr_obj_kb,
SUM (internal_object_reserved_page_count)*8 as internal_obj_kb,
SUM (version_store_reserved_page_count)*8 as version_store_kb,
SUM (unallocated_extent_page_count)*8 as freespace_kb,
SUM (mixed_extent_page_count)*8 as mixedextent_kb
FROM sys.dm_db_file_space_usage
So, if your user and internal objects are more then it clearly means that you have low TempDb space because of cursors and SQL Server internal usage (Ex: intermediate tables, Hash joins, Hash aggregation etc)
Is there a better/more efficiency way of getting a count and distinct
count of all tables in all databases?
You can use below code to get the count of all tables in all databases
DECLARE #Stats TABLE (DBNAME VARCHAR(40), NAME varchar(200), Rows INT)
INSERT INTO #Stats
EXECUTE sp_MSForEachDB
'USE ?; SELECT DB_NAME()AS DBName,
sysobjects.Name
, sysindexes.Rows
FROM
sysobjects
INNER JOIN sysindexes
ON sysobjects.id = sysindexes.id
WHERE
type = ''U''
AND sysindexes.IndId < 2'
SELECT * FROM #Stats
I have written an article on TempDb recommendation; I would suggest you to read that to understand objects which can affect TempDb and how to solve common problems of it. Ideally, your total TempDb size should be calculated based on observation which in your case > 24 GB.
** Edit 1**
If you are unsure about stats update then use below query to get count of all tables
Note: Replace databases for which you don't want stats
DECLARE #ServerStats TABLE (DatabaseName varchar(200), TableName varchar(200), RowsCount INT)
INSERT INTO #ServerStats
exec sp_msforeachdb #command1='
use #;
if ''#'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'',''ReportServer'')
begin
print ''#''
exec sp_MSforeachtable #command1=''
SELECT ''''#'''' AS DATABASENAME, ''''?'''' AS TABLENAME, COUNT(*) FROM ? ;
''
end
', #replacechar = '#'
SELECT * FROM #ServerStats
similarly you can take distinct in all tables for all databases with below query
DECLARE #ServerStatsDistinct TABLE (DatabaseName varchar(200), TableName varchar(200), RowsCount INT)
INSERT INTO #ServerStatsDistinct
exec sp_msforeachdb #command1='
use #;
if ''#'' NOT IN (''master'', ''model'', ''msdb'', ''tempdb'',''ReportServer'')
begin
print ''#''
exec sp_MSforeachtable #command1=''
SELECT ''''#'''' AS DATABASENAME, ''''?'''' AS TABLENAME, COUNT(*) FROM (
SELECT DISTINCT *
FROM ?
) a ;
''
end
', #replacechar = '#'
SELECT * FROM #ServerStatsDistinct

Is it possible to select a select in SQL without making a nvarchar variable?

I'm trying to do something like this:
--This table has a list of columns I need from #testTableTwo
CREATE TABLE #testTableOne
(
id nvarchar(250)
)
--This column has the values I need. I just don't know which columns I want.
CREATE TABLE #testTableTwo
(
one INT,
two INT,
three INT,
four int
)
INSERT INTO #testTableOne VALUES ('one'),('two'),('three')
INSERT INTO #testTableTwo VALUES (1,2,3,4)
SELECT (SELECT * FROM #testTableOne) FROM #testTableTwo
So what I want this to be is:
SELECT ONE, TWO, THREE FROM #testTableTwo
Is this possible? I know I can make a nvarchar and do some stuff with COALESCE, but I'm trying to avoid that. Is there any good way to do this?
You will have to use dynamic SQL like this.
-- First construct the fields list
DECLARE #fields varchar(500)
SET #fields = STUFF((SELECT ', ' + #testTableOne.id
FROM #testTableOne
FOR XML PATH(''),TYPE).value('.','VARCHAR(MAX)')
, 1, 2, '')
-- Next construct the actual SQL statement
DECLARE #sql nvarchar(max)
SET #sql = 'SELECT '+#fields+' FROM #testTableTwo'
-- And last but not least run it.
EXEC sp_executesql #sql

Resources