Determine segregating SQL column in known sets - sql-server

I have a known target set of records in a large Oracle DB table that I want to pull. Rather than querying them by ID though, I want to find a a column within the table that holds a value(s) that is only assigned to the known set and not assigned to any records that are not in the set, thereby giving me a value to key off of.
Example:
Target IDs: 1, 2, 3
ID Color Dir Size
1 red up S
2 red up M
3 red down L
-----------------------
4 red left S
5 blue left S
6 red left M
7 red right M
8 blue right M
In this scenario, the solution I'm looking for is the "Dir" column, as values up and down are exclusive to records in the desirable set and cover the entire set.
The table I'm working with has 80,000+ records and 100+ columns, so I'm looking for a way to perform this investigation in an automated manner, whether it be by SQL script or with tools like SSIS/SSAS 2008, Excel, PowerShell, etc. What SQL functions and/or utilities can help in this process?

I know this is a long shot, but I have a solution for you that should work in SQL Server. Of course, the caveat being that you would need to get your data from Oracle into SQL Server first.
There are a couple methods listed here:
https://dba.stackexchange.com/questions/23782/what-is-the-easiest-way-to-move-data-from-oracle-to-sql-server
If you manage to do that, you can give this a try:
IF OBJECT_ID('TestTable1', 'U') IS NOT NULL DROP TABLE TestTable1;
CREATE TABLE TestTable1 (
ID INT IDENTITY(1, 1) PRIMARY KEY,
Color VARCHAR(10),
Dir VARCHAR(10),
Size VARCHAR(10)
);
INSERT INTO TestTable1 (Color, Dir, Size)
VALUES ('red', 'up', 'S'),
('red', 'up', 'M'),
('red', 'down', 'L'),
('red', 'left', 'S'),
('blue', 'left', 'S'),
('red', 'left', 'M'),
('red', 'right', 'M'),
('blue', 'right', 'M');
DECLARE #tableName sysname = 'TestTable1';
DECLARE #targetIDs VARCHAR(MAX) = '1, 2';
-- Get all of the columns associated with the specified table:
DECLARE #targetObjectID INT = OBJECT_ID(#tableName, 'U');
DECLARE #ColumnNames TABLE (
ID INT IDENTITY(1, 1) NOT NULL PRIMARY KEY,
ColumnName sysname NOT NULL,
HasOverlap BIT NULL
);
INSERT INTO #ColumnNames (ColumnName)
SELECT c.name
FROM sys.columns c
WHERE c.object_id = #targetObjectID
AND c.name <> 'ID';
-- Define a template to use for searching column values for overlap:
DECLARE #columnTestSQL NVARCHAR(MAX) = '
WITH TargetValues AS (
-- This produces a list of values for the target IDs:
SELECT DISTINCT <ColumnName> [Val]
FROM <TableName>
WHERE ID IN (<TargetIDList>)
)
SELECT #hasOverlap_OUT = 1
FROM <TableName> t
WHERE
-- Here we check for overlap with other IDs:
t.ID NOT IN (<TargetIDList>)
AND EXISTS (
SELECT 1
FROM TargetValues tv
WHERE tv.Val = t.<ColumnName>
);
';
SET #columnTestSQL = REPLACE(#columnTestSQL, '<TableName>', #tableName);
SET #columnTestSQL = REPLACE(#columnTestSQL, '<TargetIDList>', #targetIDs);
-- Set up for loop:
DECLARE #curID INT = 1;
DECLARE #maxID INT = (SELECT MAX(ID) FROM #ColumnNames);
DECLARE #curColumnName sysname;
DECLARE #curHasOverlap BIT;
DECLARE #curSQL NVARCHAR(MAX);
-- Go through each column:
WHILE #curID <= #maxID BEGIN
-- Initialize this iteration:
SELECT
#curColumnName = cn.ColumnName,
#curHasOverlap = 0
FROM #ColumnNames cn
WHERE cn.ID = #curID;
-- Use the template to generate a dynamic SQL statement specific to the current column:
SET #curSQL = REPLACE(#columnTestSQL, '<ColumnName>', #curColumnName);
-- Execute the dynamic SQL to check for overlap:
EXEC sp_executesql
#stmt = #curSQL,
#params = N'#hasOverlap_OUT BIT OUTPUT',
#hasOverlap_OUT = #curHasOverlap OUTPUT;
-- Record the results:
UPDATE #ColumnNames
SET HasOverlap = #curHasOverlap
WHERE ID = #curID;
SET #curID += 1;
END
-- Output a list of fields with no overlap:
SELECT ColumnName
FROM #ColumnNames
WHERE HasOverlap = 0;
I have to admit I was a little thrown off by the tags on this question at first. I thought I would go ahead and post this anyway in case it helps.

Related

SQL: How to insert the result of a stored procedure to a temp table [duplicate]

How do I do a SELECT * INTO [temp table] FROM [stored procedure]? Not FROM [Table] and without defining [temp table]?
Select all data from BusinessLine into tmpBusLine works fine.
select *
into tmpBusLine
from BusinessLine
I am trying the same, but using a stored procedure that returns data, is not quite the same.
select *
into tmpBusLine
from
exec getBusinessLineHistory '16 Mar 2009'
Output message:
Msg 156, Level 15, State 1, Line 2
Incorrect syntax near the keyword
'exec'.
I have read several examples of creating a temporary table with the same structure as the output stored procedure, which works fine, but it would be nice to not supply any columns.
You can use OPENROWSET for this. Have a look. I've also included the sp_configure code to enable Ad Hoc Distributed Queries, in case it isn't already enabled.
CREATE PROC getBusinessLineHistory
AS
BEGIN
SELECT * FROM sys.databases
END
GO
sp_configure 'Show Advanced Options', 1
GO
RECONFIGURE
GO
sp_configure 'Ad Hoc Distributed Queries', 1
GO
RECONFIGURE
GO
SELECT * INTO #MyTempTable FROM OPENROWSET('SQLNCLI', 'Server=(local)\SQL2008;Trusted_Connection=yes;',
'EXEC getBusinessLineHistory')
SELECT * FROM #MyTempTable
If you want to do it without first declaring the temporary table, you could try creating a user-defined function rather than a stored procedure and make that user-defined function return a table. Alternatively, if you want to use the stored procedure, try something like this:
CREATE TABLE #tmpBus
(
COL1 INT,
COL2 INT
)
INSERT INTO #tmpBus
Exec SpGetRecords 'Params'
In SQL Server 2005 you can use INSERT INTO ... EXEC to insert the result of a stored procedure into a table. From MSDN's INSERT documentation (for SQL Server 2000, in fact):
--INSERT...EXECUTE procedure example
INSERT author_sales EXECUTE get_author_sales
This is an answer to a slightly modified version of your question. If you can abandon the use of a stored procedure for a user-defined function, you can use an inline table-valued user-defined function. This is essentially a stored procedure (will take parameters) that returns a table as a result set; and therefore will place nicely with an INTO statement.
Here's a good quick article on it and other user-defined functions. If you still have a driving need for a stored procedure, you can wrap the inline table-valued user-defined function with a stored procedure. The stored procedure just passes parameters when it calls select * from the inline table-valued user-defined function.
So for instance, you'd have an inline table-valued user-defined function to get a list of customers for a particular region:
CREATE FUNCTION CustomersByRegion
(
#RegionID int
)
RETURNS TABLE
AS
RETURN
SELECT *
FROM customers
WHERE RegionID = #RegionID
GO
You can then call this function to get what your results a such:
SELECT * FROM CustomersbyRegion(1)
Or to do a SELECT INTO:
SELECT * INTO CustList FROM CustomersbyRegion(1)
If you still need a stored procedure, then wrap the function as such:
CREATE PROCEDURE uspCustomersByRegion
(
#regionID int
)
AS
BEGIN
SELECT * FROM CustomersbyRegion(#regionID);
END
GO
I think this is the most 'hack-less' method to obtain the desired results. It uses the existing features as they were intended to be used without additional complications. By nesting the inline table-valued user-defined function in the stored procedure, you have access to the functionality in two ways. Plus! You have only one point of maintenance for the actual SQL code.
The use of OPENROWSET has been suggested, but this is not what the OPENROWSET function was intended to be used for (From Books Online):
Includes all connection information
that is required to access remote data
from an OLE DB data source. This
method is an alternative to accessing
tables in a linked server and is a
one-time, ad hoc method of connecting
and accessing remote data by using OLE
DB. For more frequent references to
OLE DB data sources, use linked
servers instead.
Using OPENROWSET will get the job done, but it will incur some additional overhead for opening up local connections and marshalling data. It also may not be an option in all cases since it requires an ad hoc query permission which poses a security risk and therefore may not be desired. Also, the OPENROWSET approach will preclude the use of stored procedures returning more than one result set. Wrapping multiple inline table-value user-defined functions in a single stored procedure can achieve this.
Select ##ServerName
EXEC sp_serveroption ##ServerName, 'DATA ACCESS', TRUE
SELECT *
INTO #tmpTable
FROM OPENQUERY(YOURSERVERNAME, 'EXEC db.schema.sproc 1')
Easiest Solution:
CREATE TABLE #temp (...);
INSERT INTO #temp
EXEC [sproc];
If you don't know the schema then you can do the following. Please
note that there are severe security risks in this method.
SELECT *
INTO #temp
FROM OPENROWSET('SQLNCLI',
'Server=localhost;Trusted_Connection=yes;',
'EXEC [db].[schema].[sproc]')
When the stored procedure returns a lot of columns and you do not want to manually "create" a temporary table to hold the result, I've found the easiest way is to go into the stored procedure and add an "into" clause on the last select statement and add 1=0 to the where clause.
Run the stored procedure once and go back and remove the SQL code you just added. Now, you'll have an empty table matching the stored procedure's result. You could either "script table as create" for a temporary table or simply insert directly into that table.
declare #temp table
(
name varchar(255),
field varchar(255),
filename varchar(255),
filegroup varchar(255),
size varchar(255),
maxsize varchar(255),
growth varchar(255),
usage varchar(255)
);
INSERT #temp Exec sp_helpfile;
select * from #temp;
If the results table of your stored proc is too complicated to type out the "create table" statement by hand, and you can't use OPENQUERY OR OPENROWSET, you can use sp_help to generate the list of columns and data types for you. Once you have the list of columns, it's just a matter of formatting it to suit your needs.
Step 1: Add "into #temp" to the output query (e.g. "select [...] into #temp from [...]").
The easiest way is to edit the output query in the proc directly. if you can't change the stored proc, you can copy the contents into a new query window and modify the query there.
Step 2: Run sp_help on the temp table. (e.g. "exec tempdb..sp_help #temp")
After creating the temp table, run sp_help on the temp table to get a list of the columns and data types including the size of varchar fields.
Step 3: Copy the data columns & types into a create table statement
I have an Excel sheet that I use to format the output of sp_help into a "create table" statement. You don't need anything that fancy, just copy and paste into your SQL editor. Use the column names, sizes, and types to construct a "Create table #x [...]" or "declare #x table [...]" statement which you can use to INSERT the results of the stored procedure.
Step 4: Insert into the newly created table
Now you'll have a query that's like the other solutions described in this thread.
DECLARE #t TABLE
(
--these columns were copied from sp_help
COL1 INT,
COL2 INT
)
INSERT INTO #t
Exec spMyProc
This technique can also be used to convert a temp table (#temp) to a table variable (#temp). While this may be more steps than just writing the create table statement yourself, it prevents manual error such as typos and data type mismatches in large processes. Debugging a typo can take more time than writing the query in the first place.
Does your stored procedure only retrieve the data or modify it too? If it's used only for retrieving, you can convert the stored procedure into a function and use the Common Table Expressions (CTEs) without having to declare it, as follows:
with temp as (
select * from dbo.fnFunctionName(10, 20)
)
select col1, col2 from temp
However, whatever needs to be retrieved from the CTE should be used in one statement only. You cannot do a with temp as ... and try to use it after a couple of lines of SQL. You can have multiple CTEs in one statement for more complex queries.
For example,
with temp1020 as (
select id from dbo.fnFunctionName(10, 20)
),
temp2030 as (
select id from dbo.fnFunctionName(20, 30)
)
select * from temp1020
where id not in (select id from temp2030)
If the OPENROWSET is causing you issues, there is another way from 2012 onwards; make use of sys.dm_exec_describe_first_result_set_for_object, as mentioned here: Retrieve column names and types of a stored procedure?
First, create this stored procedure to generate the SQL for the temporary table:
CREATE PROCEDURE dbo.usp_GetStoredProcTableDefinition(
#ProcedureName nvarchar(128),
#TableName nvarchar(128),
#SQL nvarchar(max) OUTPUT
)
AS
SET #SQL = 'CREATE TABLE ' + #tableName + ' ('
SELECT #SQL = #SQL + '['+name +'] '+ system_type_name +'' + ','
FROM sys.dm_exec_describe_first_result_set_for_object
(
OBJECT_ID(#ProcedureName),
NULL
);
--Remove trailing comma
SET #SQL = SUBSTRING(#SQL,0,LEN(#SQL))
SET #SQL = #SQL +')'
To use the procedure, call it in the following way:
DECLARE #SQL NVARCHAR(MAX)
exec dbo.usp_GetStoredProcTableDefinition
#ProcedureName='dbo.usp_YourProcedure',
#TableName='##YourGlobalTempTable',#SQL = #SQL OUTPUT
INSERT INTO ##YourGlobalTempTable
EXEC [dbo].usp_YourProcedure
select * from ##YourGlobalTempTable
Note that I'm using a global temporary table. That's because using EXEC to run the dynamic SQL creates its own session, so an ordinary temporary table would be out of scope to any subsequent code. If a global temporary table is a problem, you can use an ordinary temporary table, but any subsequent SQL would need to be dynamic, that is, also executed by the EXEC statement.
Quassnoi put me most of the way there, but one thing was missing:
****I needed to use parameters in the stored procedure.****
And OPENQUERY does not allow for this to happen:
So I found a way to work the system and also not have to make the table definition so rigid, and redefine it inside another stored procedure (and of course take the chance it may break)!
Yes, you can dynamically create the table definition returned from the stored procedure by
using the OPENQUERY statement with bogus varaiables (as long the NO RESULT SET returns the
same number of fields and in the same position as a dataset with good data).
Once the table is created, you can use exec stored procedure into the temporary table all day long.
And to note (as indicated above) you must enable data access,
EXEC sp_serveroption 'MYSERVERNAME', 'DATA ACCESS', TRUE
Code:
declare #locCompanyId varchar(8)
declare #locDateOne datetime
declare #locDateTwo datetime
set #locDateOne = '2/11/2010'
set #locDateTwo = getdate()
--Build temporary table (based on bogus variable values)
--because we just want the table definition and
--since openquery does not allow variable definitions...
--I am going to use bogus variables to get the table defintion.
select * into #tempCoAttendanceRpt20100211
FROM OPENQUERY(DBASESERVER,
'EXEC DATABASE.dbo.Proc_MyStoredProc 1,"2/1/2010","2/15/2010 3:00 pm"')
set #locCompanyId = '7753231'
insert into #tempCoAttendanceRpt20100211
EXEC DATABASE.dbo.Proc_MyStoredProc #locCompanyId,#locDateOne,#locDateTwo
set #locCompanyId = '9872231'
insert into #tempCoAttendanceRpt20100211
EXEC DATABASE.dbo.Proc_MyStoredProc #locCompanyId,#locDateOne,#locDateTwo
select * from #tempCoAttendanceRpt20100211
drop table #tempCoAttendanceRpt20100211
Thanks for the information which was provided originally...
Yes, finally I do not have to create all these bogus (strict) table defintions when using data from
another stored procedure or database, and yes you can use parameters too.
Search reference tags:
SQL 2005 stored procedure into temp table
openquery with stored procedure and variables 2005
openquery with variables
execute stored procedure into temp table
Update: this will not work with temporary tables so I had to resort to manually creating the temporary table.
Bummer notice: this will not work with temporary tables, http://www.sommarskog.se/share_data.html#OPENQUERY
Reference: The next thing is to define LOCALSERVER. It may look like a keyword in the example, but it is in fact only a name. This is how you do it:
sp_addlinkedserver #server = 'LOCALSERVER', #srvproduct = '',
#provider = 'SQLOLEDB', #datasrc = ##servername
To create a linked server, you must have the permission ALTER ANY SERVER, or be a member of any of the fixed server roles sysadmin or setupadmin.
OPENQUERY opens a new connection to SQL Server. This has some implications:
The procedure that you call with OPENQUERY cannot refer temporary tables created in the current connection.
The new connection has its own default database (defined with sp_addlinkedserver, default is master), so all object specification must include a database name.
If you have an open transaction and are holding locks when you call OPENQUERY, the called procedure can not access what you lock. That is, if you are not careful you will block yourself.
Connecting is not for free, so there is a performance penalty.
If you're lucky enough to have SQL 2012 or higher, you can use dm_exec_describe_first_result_set_for_object
I have just edited the sql provided by gotqn. Thanks gotqn.
This creates a global temp table with name same as procedure name. The temp table can later be used as required. Just don't forget to drop it before re-executing.
declare #procname nvarchar(255) = 'myProcedure',
#sql nvarchar(max)
set #sql = 'create table ##' + #procname + ' ('
begin
select #sql = #sql + '[' + r.name + '] ' + r.system_type_name + ','
from sys.procedures AS p
cross apply sys.dm_exec_describe_first_result_set_for_object(p.object_id, 0) AS r
where p.name = #procname
set #sql = substring(#sql,1,len(#sql)-1) + ')'
execute (#sql)
execute('insert ##' + #procname + ' exec ' + #procname)
end
This stored proc does the job:
CREATE PROCEDURE [dbo].[ExecIntoTable]
(
#tableName NVARCHAR(256),
#storedProcWithParameters NVARCHAR(MAX)
)
AS
BEGIN
DECLARE #driver VARCHAR(10)
DECLARE #connectionString NVARCHAR(600)
DECLARE #sql NVARCHAR(MAX)
DECLARE #rowsetSql NVARCHAR(MAX)
SET #driver = '''SQLNCLI'''
SET #connectionString =
'''server=' +
CAST(SERVERPROPERTY('ServerName') AS NVARCHAR(256)) +
COALESCE('\' + CAST(SERVERPROPERTY('InstanceName') AS NVARCHAR(256)), '') +
';trusted_connection=yes'''
SET #rowsetSql = '''EXEC ' + REPLACE(#storedProcWithParameters, '''', '''''') + ''''
SET #sql = '
SELECT
*
INTO
' + #tableName + '
FROM
OPENROWSET(' + #driver + ',' + #connectionString + ',' + #rowsetSql + ')'
EXEC (#sql)
END
GO
It's a slight rework of this: Insert stored procedure results into table so that it actually works.
If you want it to work with a temporary table then you will need to use a ##GLOBAL table and drop it afterwards.
In order to insert the first record set of a stored procedure into a temporary table you need to know the following:
only the first row set of the stored procedure can be inserted into a temporary table
the stored procedure must not execute dynamic T-SQL statement (sp_executesql)
you need to define the structure of the temporary table first
The above may look as limitation, but IMHO it perfectly makes sense - if you are using sp_executesql you can once return two columns and once ten, and if you have multiple result sets, you cannot insert them into several tables as well - you can insert maximum in two table in one T-SQL statement (using OUTPUT clause and no triggers).
So, the issue is mainly how to define the temporary table structure before performing the EXEC ... INTO ... statement.
sys.dm_exec_describe_first_result_set_for_object
sys.dm_exec_describe_first_result_set
sp_describe_first_result_set
The first works with OBJECT_ID while the second and the third works with Ad-hoc queries as well. I prefer to use the DMV instead of the sp as you can use CROSS APPLY and build the temporary table definitions for multiple procedures at the same time.
SELECT p.name, r.*
FROM sys.procedures AS p
CROSS APPLY sys.dm_exec_describe_first_result_set_for_object(p.object_id, 0) AS r;
Also, pay attention to the system_type_name field as it can be very useful. It stores the column complete definition. For, example:
smalldatetime
nvarchar(max)
uniqueidentifier
nvarchar(1000)
real
smalldatetime
decimal(18,2)
and you can use it directly in most of the cases to create the table definition.
So, I think in most of the cases (if the stored procedure match certain criteria) you can easily build dynamic statements for solving such issues (create the temporary table, insert the stored procedure result in it, do what you need with the data).
Note, that the objects above fail to define the first result set data in some cases like when dynamic T-SQL statements are executed or temporary tables are used in the stored procedure.
I'm creating a table with the following schema and data.
Create a stored procedure.
Now I know what the result of my procedure is, so I am performing the following query.
CREATE TABLE [dbo].[tblTestingTree](
[Id] [int] IDENTITY(1,1) NOT NULL,
[ParentId] [int] NULL,
[IsLeft] [bit] NULL,
[IsRight] [bit] NULL,
CONSTRAINT [PK_tblTestingTree] PRIMARY KEY CLUSTERED
(
[Id] ASC
) WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
SET IDENTITY_INSERT [dbo].[tblTestingTree] ON
INSERT [dbo].[tblTestingTree] ([Id], [ParentId], [IsLeft], [IsRight]) VALUES (1, NULL, NULL, NULL)
INSERT [dbo].[tblTestingTree] ([Id], [ParentId], [IsLeft], [IsRight]) VALUES (2, 1, 1, NULL)
INSERT [dbo].[tblTestingTree] ([Id], [ParentId], [IsLeft], [IsRight]) VALUES (3, 1, NULL, 1)
INSERT [dbo].[tblTestingTree] ([Id], [ParentId], [IsLeft], [IsRight]) VALUES (4, 2, 1, NULL)
INSERT [dbo].[tblTestingTree] ([Id], [ParentId], [IsLeft], [IsRight]) VALUES (5, 2, NULL, 1)
INSERT [dbo].[tblTestingTree] ([Id], [ParentId], [IsLeft], [IsRight]) VALUES (6, 3, 1, NULL)
INSERT [dbo].[tblTestingTree] ([Id], [ParentId], [IsLeft], [IsRight]) VALUES (7, 3, NULL, 1)
INSERT [dbo].[tblTestingTree] ([Id], [ParentId], [IsLeft], [IsRight]) VALUES (8, 4, 1, NULL)
INSERT [dbo].[tblTestingTree] ([Id], [ParentId], [IsLeft], [IsRight]) VALUES (9, 4, NULL, 1)
INSERT [dbo].[tblTestingTree] ([Id], [ParentId], [IsLeft], [IsRight]) VALUES (10, 5, 1, NULL)
SET IDENTITY_INSERT [dbo].[tblTestingTree] OFF
VALUES (10, 5, 1, NULL)
SET IDENTITY_INSERT [dbo].[tblTestingTree] On
create procedure GetDate
as
begin
select Id,ParentId from tblTestingTree
end
create table tbltemp
(
id int,
ParentId int
)
insert into tbltemp
exec GetDate
select * from tbltemp;
If the query doesn't contain parameter, use OpenQuery else use OpenRowset.
Basic thing would be to create schema as per stored procedure and insert into that table. e.g.:
DECLARE #abc TABLE(
RequisitionTypeSourceTypeID INT
, RequisitionTypeID INT
, RequisitionSourcingTypeID INT
, AutoDistOverride INT
, AllowManagerToWithdrawDistributedReq INT
, ResumeRequired INT
, WarnSupplierOnDNRReqSubmission INT
, MSPApprovalReqd INT
, EnableMSPSupplierCounterOffer INT
, RequireVendorToAcceptOffer INT
, UseCertification INT
, UseCompetency INT
, RequireRequisitionTemplate INT
, CreatedByID INT
, CreatedDate DATE
, ModifiedByID INT
, ModifiedDate DATE
, UseCandidateScheduledHours INT
, WeekEndingDayOfWeekID INT
, AllowAutoEnroll INT
)
INSERT INTO #abc
EXEC [dbo].[usp_MySp] 726,3
SELECT * FROM #abc
Code
CREATE TABLE #T1
(
col1 INT NOT NULL,
col2 NCHAR(50) NOT NULL,
col3 TEXT NOT NULL,
col4 DATETIME NULL,
col5 NCHAR(50) NULL,
col6 CHAR(2) NULL,
col6 NCHAR(100) NULL,
col7 INT NULL,
col8 NCHAR(50) NULL,
col9 DATETIME NULL,
col10 DATETIME NULL
)
DECLARE #Para1 int
DECLARE #Para2 varchar(32)
DECLARE #Para3 varchar(100)
DECLARE #Para4 varchar(15)
DECLARE #Para5 varchar (12)
DECLARE #Para6 varchar(1)
DECLARE #Para7 varchar(1)
SET #Para1 = 1025
SET #Para2 = N'6as54fsd56f46sd4f65sd'
SET #Para3 = N'XXXX\UserName'
SET #Para4 = N'127.0.0.1'
SET #Para5 = N'XXXXXXX'
SET #Para6 = N'X'
SET #Para7 = N'X'
INSERT INTO #T1
(
col1,
col2,
col3,
col4,
col5,
col6,
col6,
col7,
col8,
col9,
col10,
)
EXEC [dbo].[usp_ProcedureName] #Para1, #Para2, #Para3, #Para4, #Para5, #Para6, #Para6
I hope this helps. Please qualify as appropriate.
I found Passing Arrays/DataTables into Stored Procedures which might give you another idea on how you might go solving your problem.
The link suggests to use an Image type parameter to pass into the stored procedure. Then in the stored procedure, the image is transformed into a table variable containing the original data.
Maybe there is a way this can be used with a temporary table.
I met the same problem and here is what I did for this from Paul's suggestion. The main part is here is to use NEWID() to avoid multiple users run the store procedures/scripts at the same time, the pain for global temporary table.
DECLARE #sql varchar(max) = '',
#tmp_global_table varchar(255) = '##global_tmp_' + CONVERT(varchar(36), NEWID())
SET #sql = #sql + 'select * into [' + #tmp_global_table + '] from YOURTABLE'
EXEC(#sql)
EXEC('SELECT * FROM [' + #tmp_global_table + ']')
Another method is to create a type and use PIPELINED to then pass back your object. This is limited to knowing the columns however. But it has the advantage of being able to do:
SELECT *
FROM TABLE(CAST(f$my_functions('8028767') AS my_tab_type))
This can be done in SQL Server 2014+ provided the stored procedure only returns one table. If anyone finds a way of doing this for multiple tables I'd love to know about it.
DECLARE #storedProcname NVARCHAR(MAX) = ''
SET #storedProcname = 'myStoredProc'
DECLARE #strSQL AS VARCHAR(MAX) = 'CREATE TABLE myTableName '
SELECT #strSQL = #strSQL+STUFF((
SELECT ',' +name+' ' + system_type_name
FROM sys.dm_exec_describe_first_result_set_for_object (OBJECT_ID(#storedProcname),0)
FOR XML PATH('')
),1,1,'(') + ')'
EXEC (#strSQL)
INSERT INTO myTableName
EXEC ('myStoredProc #param1=1, #param2=2')
SELECT * FROM myTableName
DROP TABLE myTableName
This pulls the definition of the returned table from system tables, and uses that to build the temp table for you. You can then populate it from the stored procedure as stated before.
There are also variants of this that work with Dynamic SQL too.
After searching around I found a way to create a temp table dynamically for any stored procedure without using OPENROWSET or OPENQUERY using a generic schema of Stored Procedure's result definition especially when you are not database Administrator.
Sql server has a buit-in proc sp_describe_first_result_set that can provide you with schema of any procedures resultset. I created a schema table from results of this procedure and manually set all the field to NULLABLE.
declare #procname varchar(100) = 'PROCEDURENAME' -- your procedure name
declare #param varchar(max) = '''2019-06-06''' -- your parameters
declare #execstr nvarchar(max) = N'exec ' + #procname
declare #qry nvarchar(max)
-- Schema table to store the result from sp_describe_first_result_set.
create table #d
(is_hidden bit NULL, column_ordinal int NULL, name sysname NULL, is_nullable bit NULL, system_type_id int NULL, system_type_name nvarchar(256) NULL,
max_length smallint NULL, precision tinyint NULL, scale tinyint NULL, collation_name sysname NULL, user_type_id int NULL, user_type_database sysname NULL,
user_type_schema sysname NULL,user_type_name sysname NULL,assembly_qualified_type_name nvarchar(4000),xml_collection_id int NULL,xml_collection_database sysname NULL,
xml_collection_schema sysname NULL,xml_collection_name sysname NULL,is_xml_document bit NULL,is_case_sensitive bit NULL,is_fixed_length_clr_type bit NULL,
source_server sysname NULL,source_database sysname NULL,source_schema sysname NULL,source_table sysname NULL,source_column sysname NULL,is_identity_column bit NULL,
is_part_of_unique_key bit NULL,is_updateable bit NULL,is_computed_column bit NULL,is_sparse_column_set bit NULL,ordinal_in_order_by_list smallint NULL,
order_by_list_length smallint NULL,order_by_is_descending smallint NULL,tds_type_id int NULL,tds_length int NULL,tds_collation_id int NULL,
tds_collation_sort_id tinyint NULL)
-- Get result set definition of your procedure
insert into #d
EXEC sp_describe_first_result_set #exestr, NULL, 0
-- Create a query to generate and populate a global temp table from above results
select
#qry = 'Create table ##t(' +
stuff(
(select ',' + name + ' '+ system_type_name + ' NULL'
from #d d For XML Path, TYPE)
.value(N'.[1]', N'nvarchar(max)')
, 1,1,'')
+ ')
insert into ##t
Exec '+#procname+' ' + #param
Exec sp_executesql #qry
-- Use below global temp table to query the data as you may
select * from ##t
-- **WARNING** Don't forget to drop the global temp table ##t.
--drop table ##t
drop table #d
Developed and tested on Sql Server version - Microsoft SQL Server 2016 (RTM) - 13.0.1601.5(Build 17134:)
You can tweak the schema for your SQL server version that you are using (if needed).
It's a simple 2 step process:
- create a temporary table
- Insert into the temporary table.
Code to perform the same:
CREATE TABLE #tempTable (Column1 int, Column2 varchar(max));
INSERT INTO #tempTable
EXEC [app].[Sproc_name]
#param1 = 1,
#param2 =2;
If you know the parameters that are being passed and if you don't have access to make sp_configure, then edit the stored procedure with these parameters and the same can be stored in a ##global table.
A few years late to the question, but I needed something like this for some quick and dirty code generation. I believe as others have stated it is just easier to define the temp table up front, but this method should work for simple stored procedure queries or sql statments.
This will be a little convoluted, but it borrows from the contributors here as well as Paul White's solution from DBA Stack Exchange Get stored procedure result column-types. Again, to reiterate this approach & example is not designed for processes in a multi user environment. In this case the table definition is being set for a short time in a global temp table for reference by a code generation template process.
I haven't fully tested this so there may be caveats so you may want to go to the MSDN link in Paul White's answer. This applies to SQL 2012 and higher.
First use the stored procedure sp_describe_first_result_set which resembles Oracle's describe.
This will evaluate the first row of the first result set so if your stored procedure or statement returns multiple queries it will only describe the first result.
I created a stored proc to break down the tasks that returns a single field to select from to create the temp table definition.
CREATE OR ALTER PROCEDURE [dbo].[sp_GetTableDefinitionFromSqlBatch_DescribeFirstResultSet]
(
#sql NVARCHAR(4000)
,#table_name VARCHAR(100)
,#TableDefinition NVARCHAR(MAX) OUTPUT
)
AS
BEGIN
SET NOCOUNT ON
DECLARE #TempTableDefinition NVARCHAR(MAX)
DECLARE #NewLine NVARCHAR(4) = CHAR(13)+CHAR(10)
DECLARE #ResultDefinition TABLE ( --The View Definition per MSDN
is_hidden bit NOT NULL
, column_ordinal int NOT NULL
, [name] sysname NULL
, is_nullable bit NOT NULL
, system_type_id int NOT NULL
, system_type_name nvarchar(256) NULL
, max_length smallint NOT NULL
, [precision] tinyint NOT NULL
, scale tinyint NOT NULL
, collation_name sysname NULL
, user_type_id int NULL
, user_type_database sysname NULL
, user_type_schema sysname NULL
, user_type_name sysname NULL
, assembly_qualified_type_name nvarchar(4000)
, xml_collection_id int NULL
, xml_collection_database sysname NULL
, xml_collection_schema sysname NULL
, xml_collection_name sysname NULL
, is_xml_document bit NOT NULL
, is_case_sensitive bit NOT NULL
, is_fixed_length_clr_type bit NOT NULL
, source_server sysname NULL
, source_database sysname NULL
, source_schema sysname NULL
, source_table sysname NULL
, source_column sysname NULL
, is_identity_column bit NULL
, is_part_of_unique_key bit NULL
, is_updateable bit NULL
, is_computed_column bit NULL
, is_sparse_column_set bit NULL
, ordinal_in_order_by_list smallint NULL
, order_by_is_descending smallint NULL
, order_by_list_length smallint NULL
, tds_type_id int NOT NULL
, tds_length int NOT NULL
, tds_collation_id int NULL
, tds_collation_sort_id tinyint NULL
)
--Insert the description into table variable
INSERT #ResultDefinition
EXEC sp_describe_first_result_set #sql
--Now Build the string to create the table via union select statement
;WITH STMT AS (
SELECT N'CREATE TABLE ' + #table_name + N' (' AS TextVal
UNION ALL
SELECT
CONCAT(
CASE column_ordinal
WHEN 1 THEN ' ' ELSE ' , ' END --Determines if comma should precede
, QUOTENAME([name]) , ' ', system_type_name -- Column Name and SQL TYPE
,CASE is_nullable
WHEN 0 THEN ' NOT NULL' ELSE ' NULL' END --NULLABLE CONSTRAINT
) AS TextVal
FROM #ResultDefinition WHERE is_hidden = 0 -- May not be needed
UNION ALL
SELECT N');' + #NewLine
)
--Now Combine the rows to a single String
SELECT #TempTableDefinition = COALESCE (#TempTableDefinition + #NewLine + TextVal, TextVal) FROM STMT
SELECT #TableDefinition = #TempTableDefinition
END
The conundrum is that you need to use a global table, but you need to make it unique enough
so you can drop and create from it frequently without worrying about a collision.
In the example I used a Guid (FE264BF5_9C32_438F_8462_8A5DC8DEE49E) for the global variable replacing the hyphens with underscore
DECLARE #sql NVARCHAR(4000) = N'SELECT ##SERVERNAME as ServerName, GETDATE() AS Today;'
DECLARE #GlobalTempTable VARCHAR(100) = N'##FE264BF5_9C32_438F_8462_8A5DC8DEE49E_MyTempTable'
--#sql can be a stored procedure name like dbo.foo without parameters
DECLARE #TableDef NVARCHAR(MAX)
DROP TABLE IF EXISTS #MyTempTable
DROP TABLE IF EXISTS ##FE264BF5_9C32_438F_8462_8A5DC8DEE49E_MyTempTable
EXEC [dbo].[sp_GetTableDefinitionFromSqlBatch_DescribeFirstResultSet]
#sql, #GlobalTempTable, #TableDef OUTPUT
--Creates the global table ##FE264BF5_9C32_438F_8462_8A5DC8DEE49E_MyTempTable
EXEC sp_executesql #TableDef
--Now Call the stored procedure, SQL Statement with Params etc.
INSERT ##FE264BF5_9C32_438F_8462_8A5DC8DEE49E_MyTempTable
EXEC sp_executesql #sql
--Select the results into your undefined Temp Table from the Global Table
SELECT *
INTO #MyTempTable
FROM ##FE264BF5_9C32_438F_8462_8A5DC8DEE49E_MyTempTable
SELECT * FROM #MyTempTable
DROP TABLE IF EXISTS #MyTempTable
DROP TABLE IF EXISTS ##FE264BF5_9C32_438F_8462_8A5DC8DEE49E_MyTempTable
Again, I have only tested it with simple stored procedure queries and simple queries so your mileage may vary. Hope this helps someone.
Here is my T-SQL with parameters
--require one time execution if not configured before
sp_configure 'Show Advanced Options', 1
GO
RECONFIGURE
GO
--require one time execution if not configured before
sp_configure 'Ad Hoc Distributed Queries', 1
GO
RECONFIGURE
GO
--the query
DECLARE #param1 int = 1, #param2 int = 2
DECLARE #SQLStr varchar(max) = 'SELECT * INTO #MyTempTable
FROM OPENROWSET(''SQLNCLI'',
''Server=ServerName;Database=DbName;Trusted_Connection=yes'',
''exec StoredProcedureName '+ CAST(#param1 AS varchar(15)) +','+ CAST(#param2 AS varchar(15)) +''') AS a ;
select * from #MyTempTable;
drop table #MyTempTable
';
EXECUTE(#SQLStr);
Well, you do have to create a temp table, but it doesn't have to have the right schema....I've created a stored procedure that modifies an existing temp table so that it has the required columns with the right data type and order (dropping all existing columns, adding new columns):
GO
create procedure #TempTableForSP(#tableId int, #procedureId int)
as
begin
declare #tableName varchar(max) = (select name
from tempdb.sys.tables
where object_id = #tableId
);
declare #tsql nvarchar(max);
declare #tempId nvarchar(max) = newid();
set #tsql = '
declare #drop nvarchar(max) = (select ''alter table tempdb.dbo.' + #tableName
+ ' drop column '' + quotename(c.name) + '';''+ char(10)
from tempdb.sys.columns c
where c.object_id = ' +
cast(#tableId as varchar(max)) + '
for xml path('''')
)
alter table tempdb.dbo.' + #tableName + ' add ' + QUOTENAME(#tempId) + ' int;
exec sp_executeSQL #drop;
declare #add nvarchar(max) = (
select ''alter table ' + #tableName
+ ' add '' + name
+ '' '' + system_type_name
+ case when d.is_nullable=1 then '' null '' else '''' end
+ char(10)
from sys.dm_exec_describe_first_result_set_for_object('
+ cast(#procedureId as varchar(max)) + ', 0) d
order by column_ordinal
for xml path(''''))
execute sp_executeSQL #add;
alter table ' + #tableName + ' drop column ' + quotename(#tempId) + ' ';
execute sp_executeSQL #tsql;
end
GO
create table #exampleTable (pk int);
declare #tableId int = object_Id('tempdb..#exampleTable')
declare #procedureId int = object_id('examplestoredProcedure')
exec #TempTableForSP #tableId, #procedureId;
insert into #exampleTable
exec examplestoredProcedure
Note this won't work if sys.dm_exec_describe_first_result_set_for_object can't determine the results of the stored procedure (for instance if it uses a temp table).
If you let dynamic SQL create a temp table, this table is owned by the Dynamic SQL connection, as opposed to the connection your stored procedure is called from.
DECLARE #COMMA_SEPARATED_KEYS varchar(MAX);
DROP TABLE IF EXISTS KV;
CREATE TABLE KV (id_person int, mykey varchar(30), myvalue int);
INSERT INTO KV VALUES
(1, 'age', 16),
(1, 'weight', 63),
(1, 'height', 175),
(2, 'age', 26),
(2, 'weight', 83),
(2, 'height', 185);
WITH cte(mykey) AS (
SELECT DISTINCT mykey FROM KV
)
SELECT #COMMA_SEPARATED_KEYS=STRING_AGG(mykey,',') FROM cte;
SELECT #COMMA_SEPARATED_KEYS AS keys;
DECLARE #ExecuteExpression varchar(MAX);
DROP TABLE IF EXISTS #Pivoted;
SET #ExecuteExpression = N'
SELECT *
INTO #Pivoted
FROM
(
SELECT
mykey,
myvalue,
id_person
FROM KV
) AS t
PIVOT(
MAX(t.myvalue)
FOR mykey IN (COMMA_SEPARATED_KEYS)
) AS pivot_table;
';
SET #ExecuteExpression = REPLACE(#ExecuteExpression, 'COMMA_SEPARATED_KEYS', #COMMA_SEPARATED_KEYS);
EXEC(#ExecuteExpression);
SELECT * FROM #Pivoted;
Msg 208, Level 16, State 0
Invalid object name '#Pivoted'.
This is because #Pivoted is owned by the Dynamic SQL connection. So the last instruction
SELECT * FROM #Pivoted
fails.
One way to not face this issue is to make sure all references to #Pivoted are made from inside the dynamic query itself:
DECLARE #COMMA_SEPARATED_KEYS varchar(MAX);
DROP TABLE IF EXISTS KV;
CREATE TABLE KV (id_person int, mykey varchar(30), myvalue int);
INSERT INTO KV VALUES
(1, 'age', 16),
(1, 'weight', 63),
(1, 'height', 175),
(2, 'age', 26),
(2, 'weight', 83),
(2, 'height', 185);
WITH cte(mykey) AS (
SELECT DISTINCT mykey FROM KV
)
SELECT #COMMA_SEPARATED_KEYS=STRING_AGG(mykey,',') FROM cte;
SELECT #COMMA_SEPARATED_KEYS AS keys;
DECLARE #ExecuteExpression varchar(MAX);
DROP TABLE IF EXISTS #Pivoted;
SET #ExecuteExpression = N'
SELECT *
INTO #Pivoted
FROM
(
SELECT
mykey,
myvalue,
id_person
FROM KV
) AS t
PIVOT(
MAX(t.myvalue)
FOR mykey IN (COMMA_SEPARATED_KEYS)
) AS pivot_table;
SELECT * FROM #Pivoted;
';
SET #ExecuteExpression = REPLACE(#ExecuteExpression, 'COMMA_SEPARATED_KEYS', #COMMA_SEPARATED_KEYS);
EXEC(#ExecuteExpression);
First, modify your stored procedure to save the end results in to a temp table. By doing this we are creating a table matching with the SP output fields. And then have a select statement to save that temp table to a any table name. Then execute the SP as explained in step 2
Step 1: modify your stored procedure to save the end results in to a temp table
[your stored procedure]
into #table_temp //this will insert the data to a temp table
from #table_temp
select * into SP_Output_Table_1 from #table_temp //this will save data to a actual table
Step 2: Execute the SP as below that will insert records to your table
Insert SP_Output_Table_1
EXE You_SP_Nane #Parameter1 = 52, #parameter2 =1

How can I extract all literal strings from text in SQL server?

I need to extract all the literal strings from my SQL server stored procedures and put them in a table. I tried to use Jeff Moden's and other string splitters, but all the code in between the quoted strings gets in the way and is hard to identify as not useful. The most simple example of what I am trying to accomplish:
input = "SELECT * FROM table WHERE column IN ('cool','super', 'awesome')
output table:
cool
super
awesome
Here is an example using the sp_help stored procedure.
USE master
SELECT * FROM sys.system_sql_modules where OBJECT_NAME(object_id) = 'sp_help'
I want the output to be ONLY the single quoted items contained in sp_help - here is the snippet of sp_help.
create procedure sys.sp_help
#objname nvarchar(776) = NULL -- object name we're after
as
-- PRELIMINARY
set nocount on
declare #dbname sysname
,#no varchar(35), #yes varchar(35), #none varchar(35)
select #no = 'no', #yes = 'yes', #none = 'none'
-- If no #objname given, give a little info about all objects.
if #objname is null
begin
-- DISPLAY ALL SYSOBJECTS --
select
'Name' = o.name,
'Owner' = user_name(ObjectProperty( object_id, 'ownerid')),
'Object_type' = substring(v.name,5,31)
from sys.all_objects o, master.dbo.spt_values v
where o.type = substring(v.name,1,2) collate database_default and v.type = 'O9T'
order by [Owner] asc, Object_type desc, Name asc
The output would be to a table:
- no
- yes
- none
- Name
- Owner
- ownerid
- Object_type
- O9T
Does anyone have anything like this already or can adapt one of the common string splitters to ignore the stuff in between opposing quoted items?
Here's a small test I made. The find_next function is a replacement for charindex. The difference is that when it's looking the end of the string, it will skip '' to prevent that from ending the string. If that happens it will call itself recursively:
CREATE FUNCTION find_next (#str nvarchar(MAX), #start int, #mode int)
returns bigint
as
begin
declare #pos int
set #pos = charindex('''', #str, #start)
if (#pos = 0) return 0
if (#mode = 1) return #pos
if (substring(#str, #pos+1,1)='''') return dbo.find_next(#str, #start+2, #mode)
return #pos
end
And a recursive CTE based function to find the strings:
CREATE FUNCTION find_strs (#str nvarchar(MAX))
RETURNS TABLE AS
RETURN
WITH CTE AS (
select
convert(bigint, 1) as start,
charindex('''', #str) as stop,
0 as type
UNION ALL
select
stop + 1,
dbo.find_next(#str, stop + 1, type),
(type + 1) % 2
FROM CTE
WHERE stop > 0
)
SELECT substring(#str, start, CASE WHEN stop > 0 THEN stop - start ELSE 0 END) as Str
FROM CTE
WHERE stop > 0 and type = 1
Simple example in SQL Fiddle

How can I avoid calling a stored procedure from a UDF in SQL Server

Before I start, I know you can't call a stored procedure from a UDF and I know that there are various "reasons" for this (none that make much sense to me though, tbh it just sounds like laziness on Microsoft's part).
I am more interested in how I can design a system to get round this flaw in SQL Server.
This is a quick overview of the system I currently have:
I have a dynamic report generator where users specify data items, operators (=, <, !=, etc.) and filter values. These are used to build up "rules" with one or more filters, e.g. I might have a rule that has two filters "Category < 12" and "Location != 'York'";
there are thousands and thousands of these "rules", some of them have many, many filters;
the output from each of these rules is a statuory report that always has exactly the same "shape", i.e. the same columns/ data types. Basically these reports produce lists of tonnages and materials;
I have a scalar-valued function that generates Dynamic SQL for a specified rule, returning this as a VARCHAR(MAX);
I have a stored procedure that is called to run a specific rule, it calls the UDF to generate the Dynamic SQL, runs this and returns the results (this used to just return the results but now I store the output in process-keyed tables to make the data easier to share and so I return a handle to this data instead);
I have a stored procedure that is called to run all the rules for a particular company, so it makes a list of the rules to run, runs them sequentially and then merges the results together as output.
So this all works perfectly.
Now I want one final thing, a report that runs the company summary and then applies costs to the tonnages/ materials to result in a cost report. This seemed such a simple requirement when I started on this last week :'(
My report has to be a table-valued function for it to work with the report broker system I have already written. If I write it as a stored procedure then it will not be run through my report broker which means that it will not be controlled, i.e. I won't know who ran the report and when.
But I can't call a stored procedure from within a table-valued function and the two obvious ways to handle this are as follows:
Get the SQL to create the output, run it and suck up the results.
--Method #1
WHILE #RuleIndex <= #MaxRuleIndex
BEGIN
DECLARE #DSFId UNIQUEIDENTIFIER;
SELECT #DSFId = [GUID] FROM NewGUID; --this has to be deterministic, it isn't but the compiler thinks it is and that's good enough :D
DECLARE #RuleId UNIQUEIDENTIFIER;
SELECT #RuleId = DSFRuleId FROM #DSFRules WHERE DSFRuleIndex = #RuleIndex;
DECLARE #SQL VARCHAR(MAX);
--Get the SQL
SELECT #SQL = DSF.DSFEngine(#ServiceId, #MemberId, #LocationId, #DSFYear, NULL, NULL, NULL, NULL, #DSFId, #RuleId);
--Run it
EXECUTE(#SQL);
--Copy the data out of the results table into our local copy
INSERT INTO
#DSFResults
SELECT
TableId, TableCode, TableName, RowId, RowName, LocationCode, LocationName, ProductCode, ProductName, PackagingGroupCode, PackagingGroupName, LevelName, WeightSource, Quantity, Paper, Glass, Aluminium, Steel, Plastic, Wood, Other, 0 AS General
FROM
DSF.DSFPackagingResults
WHERE
DSFId = #DSFId
AND RuleId = #RuleId;
SELECT #RuleIndex = #RuleIndex + 1;
END;
Call the report directly
--Method #2
WHILE #RuleIndex <= #MaxRuleIndex
BEGIN
DECLARE #DSFId UNIQUEIDENTIFIER;
SELECT #DSFId = [GUID] FROM NewGUID; --this has to be deterministic, it isn't but the compiler thinks it is :D
DECLARE #RuleId UNIQUEIDENTIFIER;
SELECT #RuleId = DSFRuleId FROM #DSFRules WHERE DSFRuleIndex = #RuleIndex;
DECLARE #SQL VARCHAR(MAX);
--Run the report
EXECUTE ExecuteDSFRule #ServiceId, #MemberId, #LocationId, #DSFYear, NULL, NULL, NULL, #RuleId, #DSFId, 2;
--Copy the data out of the results table into our local copy
INSERT INTO
#DSFResults
SELECT
TableId, TableCode, TableName, RowId, RowName, LocationCode, LocationName, ProductCode, ProductName, PackagingGroupCode, PackagingGroupName, LevelName, WeightSource, Quantity, Paper, Glass, Aluminium, Steel, Plastic, Wood, Other, 0 AS General
FROM
DSF.DSFPackagingResults
WHERE
DSFId = #DSFId
AND RuleId = #RuleId;
SELECT #RuleIndex = #RuleIndex + 1;
END;
I can think of the following workarounds (none of which are particularly satisfactory):
rewrite some of this in CLR (but this is just a whole lot of hassle to break the rules);
use a stored procedure to produce my report (but this means I lose control of the execution unless I develop a new system for this SINGLE report, different to the dozens of existing reports that all work fine);
split execution from reporting, so I have one process to execute the report and another that just picks up the output (but no way to tell when the report has completed without more work);
wait until Microsoft see sense and allow execution of stored procedures from UDFs.
Any other ideas out there?
Edit 3-May-2013, here is a (very) simple example of how this hangs together:
--Data to be reported
CREATE TABLE DataTable (
MemberId INT,
ProductId INT,
ProductSize VARCHAR(50),
Imported INT,
[Weight] NUMERIC(19,2));
INSERT INTO DataTable VALUES (1, 1, 'Large', 0, 5.4);
INSERT INTO DataTable VALUES (1, 2, 'Large', 1, 6.2);
INSERT INTO DataTable VALUES (1, 3, 'Medium', 0, 2.3);
INSERT INTO DataTable VALUES (1, 4, 'Small', 1, 1.9);
INSERT INTO DataTable VALUES (1, 5, 'Small', 0, 0.7);
INSERT INTO DataTable VALUES (1, 6, 'Small', 1, 1.2);
--Report Headers
CREATE TABLE ReportsTable (
ReportHandle INT,
ReportName VARCHAR(50));
INSERT INTO ReportsTable VALUES (1, 'Large Products');
INSERT INTO ReportsTable VALUES (2, 'Imported Small Products');
--Report Detail
CREATE TABLE ReportsDetail (
ReportHandle INT,
ReportDetailHandle INT,
DatabaseColumn VARCHAR(50),
DataType VARCHAR(50),
Operator VARCHAR(3),
FilterValue VARCHAR(50));
INSERT INTO ReportsDetail VALUES (1, 1, 'ProductSize', 'VARCHAR', '=', 'Large');
INSERT INTO ReportsDetail VALUES (2, 1, 'Imported', 'INT', '=', '1');
INSERT INTO ReportsDetail VALUES (2, 1, 'ProductSize', 'VARCHAR', '=', 'Small');
GO
CREATE FUNCTION GenerateReportSQL (
#ReportHandle INT)
RETURNS VARCHAR(MAX)
AS
BEGIN
DECLARE #SQL VARCHAR(MAX);
SELECT #SQL = 'SELECT SUM([Weight]) FROM DataTable WHERE 1=1 ';
DECLARE #Filters TABLE (
FilterIndex INT,
DatabaseColumn VARCHAR(50),
DataType VARCHAR(50),
Operator VARCHAR(3),
FilterValue VARCHAR(50));
INSERT INTO #Filters SELECT ROW_NUMBER() OVER (ORDER BY DatabaseColumn), DatabaseColumn, DataType, Operator, FilterValue FROM ReportsDetail WHERE ReportHandle = #ReportHandle;
DECLARE #FilterIndex INT = NULL;
SELECT TOP 1 #FilterIndex = FilterIndex FROM #Filters;
WHILE #FilterIndex IS NOT NULL
BEGIN
SELECT TOP 1 #SQL = #SQL + ' AND ' + DatabaseColumn + ' ' + Operator + ' ' + CASE WHEN DataType = 'VARCHAR' THEN '''' ELSE '' END + FilterValue + CASE WHEN DataType = 'VARCHAR' THEN '''' ELSE '' END FROM #Filters WHERE FilterIndex = #FilterIndex;
DELETE FROM #Filters WHERE FilterIndex = #FilterIndex;
SELECT #FilterIndex = NULL;
SELECT TOP 1 #FilterIndex = FilterIndex FROM #Filters;
END;
RETURN #SQL;
END;
GO
CREATE PROCEDURE ExecuteReport (
#ReportHandle INT)
AS
BEGIN
--Get the SQL
DECLARE #SQL VARCHAR(MAX);
SELECT #SQL = dbo.GenerateReportSQL(#ReportHandle);
EXECUTE (#SQL);
END;
GO
--Test
EXECUTE ExecuteReport 1;
EXECUTE ExecuteReport 2;
SELECT dbo.GenerateReportSQL(1);
SELECT dbo.GenerateReportSQL(2);
GO
--What I really want
CREATE FUNCTION RunReport (
#ReportHandle INT)
RETURNS #Results TABLE ([Weight] NUMERIC(19,2))
AS
BEGIN
INSERT INTO #Results EXECUTE ExecuteReport #ReportHandle;
RETURN;
END;
--Invalid use of a side-effecting operator 'INSERT EXEC' within a function
If I was in your situation, I wouldn't try to hack anything. I would setup the objects like this:
CREATE TABLE [dbo].[ReportCollection] (
[ReportCollectionID] int,
[ReportID] int
)
CREATE TABLE [dbo].[ReportResult] (
[ReportID] int,
[LocationCode] int,
[LocationName] nvarchar(max)
)
CREATE PROCEDURE [dbo].[usp_ExecuteReport] (
#ReportID int
)
AS
INSERT [dbo].[ReportResult]
SELECT #ReportID, 1, N'StackOverflow'
END
CREATE FUNCTION [dbo].[udf_RetrieveReportCollectionResults] (
#ReportCollectionID int
)
RETURNS #Results TABLE ([ReportID], [LocationCode], [LocationName])
AS
BEGIN
SELECT *
FROM [dbo].[ReportResult] rr
JOIN [dbo].[ReportCollection] rc
ON rr.ReportID = rc.ReportID
WHERE rc.ReportCollectionID = #ReportCollectionID
END
And use them like this:
INSERT [dbo].[ReportCollection] VALUES (1, 1)
INSERT [dbo].[ReportCollection] VALUES (1, 2)
EXEC [dbo].[usp_ExecuteReport] #ReportID = 1
EXEC [dbo].[usp_ExecuteReport] #ReportID = 2
SELECT * FROM [dbo].[udf_RetrieveReportCollectionResults](1)
Each time you run your reports, start a new collection. Your application should kick off all of the reports and consolidate the results afterward.
--
If you really wanted to call a stored procedure from a udf (please don't), do a search on xp_cmdshell.
If you really want this working as a function then the least hacky way would be CLR integration.
You don't have to redo everything - just write a CLR wrapper function that calls the stored procedure & returns the stored procs result set as it's own.1
This way all your current SQL development is untouched.

How to detect interface break between stored procedure

I am working on a large project with a lot of stored procedures. I came into the following situation where a developer modified the arguments of a stored procedure which was called by another stored procedure.
Unfortunately, nothing prevents the ALTER PROC to complete.
Is there a way to perform those checks afterwards ?
What would be the guidelines to avoid getting into that kind of problems ?
Here is a sample code to reproduce this behavior :
CREATE PROC Test1 #arg1 int
AS
BEGIN
PRINT CONVERT(varchar(32), #arg1)
END
GO
CREATE PROC Test2 #arg1 int
AS
BEGIN
DECLARE #arg int;
SET #arg = #arg1+1;
EXEC Test1 #arg;
END
GO
EXEC Test2 1;
GO
ALTER PROC Test1 #arg1 int, #arg2 int AS
BEGIN
PRINT CONVERT(varchar(32), #arg1)
PRINT CONVERT(varchar(32), #arg2)
END
GO
EXEC Test2 1;
GO
DROP PROC Test2
DROP PROC Test1
GO
Sql server 2005 has a system view sys.sql_dependencies that tracks dependencies. Unfortunately, it's not all that reliable (For more info, see this answer). Oracle, however, is much better in that regard. So you could switch. There's also a 3rd party vendor, Redgate, who has Sql Dependency Tracker. Never tested it myself but there is a trial version available.
I have the same problem so I implemented my poor man's solution by creating a stored procedure that can search for strings in all the stored procedures and views in the current database. By searching on the name of the changed stored procedure I can (hopefully) find EXEC calls.
I used this on sql server 2000 and 2008 so it probably also works on 2005. (Note : #word1, #word2, etc must all be present but that can easily be changed in the last SELECT if you have different needs.)
CREATE PROCEDURE [dbo].[findWordsInStoredProceduresViews]
#word1 nvarchar(4000) = null,
#word2 nvarchar(4000) = null,
#word3 nvarchar(4000) = null,
#word4 nvarchar(4000) = null,
#word5 nvarchar(4000) = null
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
-- create temp table
create table #temp
(
id int identity(1,1),
Proc_id INT,
Proc_Name SYSNAME,
Definition NTEXT
)
-- get the names of the procedures that meet our criteria
INSERT #temp(Proc_id, Proc_Name)
SELECT id, OBJECT_NAME(id)
FROM syscomments
WHERE OBJECTPROPERTY(id, 'IsProcedure') = 1 or
OBJECTPROPERTY(id, 'IsView') = 1
GROUP BY id, OBJECT_NAME(id)
-- initialize the NTEXT column so there is a pointer
UPDATE #temp SET Definition = ''
-- declare local variables
DECLARE
#txtPval binary(16),
#txtPidx INT,
#curText NVARCHAR(4000),
#counterId int,
#maxCounterId int,
#counterIdInner int,
#maxCounterIdInner int
-- set up a double while loop to get the data from syscomments
select #maxCounterId = max(id)
from #temp t
create table #tempInner
(
id int identity(1,1),
curName SYSNAME,
curtext ntext
)
set #counterId = 0
WHILE (#counterId < #maxCounterId)
BEGIN
set #counterId = #counterId + 1
insert into #tempInner(curName, curtext)
SELECT OBJECT_NAME(s.id), text
FROM syscomments s
INNER JOIN #temp t
ON s.id = t.Proc_id
WHERE t.id = #counterid
ORDER BY s.id, colid
select #maxCounterIdInner = max(id)
from #tempInner t
set #counterIdInner = 0
while (#counterIdInner < #maxCounterIdInner)
begin
set #counterIdInner = #counterIdInner + 1
-- get the pointer for the current procedure name / colid
SELECT #txtPval = TEXTPTR(Definition)
FROM #temp
WHERE id = #counterId
-- find out where to append the #temp table's value
SELECT #txtPidx = DATALENGTH(Definition)/2
FROM #temp
WHERE id = #counterId
select #curText = curtext
from #tempInner
where id = #counterIdInner
-- apply the append of the current 8KB chunk
UPDATETEXT #temp.definition #txtPval #txtPidx 0 #curtext
end
truncate table #tempInner
END
-- check our filter
SELECT Proc_Name, Definition
FROM #temp t
WHERE (#word1 is null or definition LIKE '%' + #word1 + '%') AND
(#word2 is null or definition LIKE '%' + #word2 + '%') AND
(#word3 is null or definition LIKE '%' + #word3 + '%') AND
(#word4 is null or definition LIKE '%' + #word4 + '%') AND
(#word5 is null or definition LIKE '%' + #word5 + '%')
ORDER BY Proc_Name
-- clean up
DROP TABLE #temp
DROP TABLE #tempInner
END
You can use sp_refreshsqlmodule to attempt to re-validate SPs (this also updates dependencies), but it won't validate this particular scenario with parameters at the caller level (it will validate things like invalid columns in tables and views).
http://www.mssqltips.com/tip.asp?tip=1294 has a number of techniques, including sp_depends
Dependency information is stored in the SQL Server metadata, including parameter columns/types for each SP and function, but it isn't obvious how to validate all the calls, but it is possible to locate them and inspect them.

Is there a way to make a TSQL variable constant?

Is there a way to make a TSQL variable constant?
No, but you can create a function and hardcode it in there and use that.
Here is an example:
CREATE FUNCTION fnConstant()
RETURNS INT
AS
BEGIN
RETURN 2
END
GO
SELECT dbo.fnConstant()
One solution, offered by Jared Ko is to use pseudo-constants.
As explained in SQL Server: Variables, Parameters or Literals? Or… Constants?:
Pseudo-Constants are not variables or parameters. Instead, they're simply views with one row, and enough columns to support your constants. With these simple rules, the SQL Engine completely ignores the value of the view but still builds an execution plan based on its value. The execution plan doesn't even show a join to the view!
Create like this:
CREATE SCHEMA ShipMethod
GO
-- Each view can only have one row.
-- Create one column for each desired constant.
-- Each column is restricted to a single value.
CREATE VIEW ShipMethod.ShipMethodID AS
SELECT CAST(1 AS INT) AS [XRQ - TRUCK GROUND]
,CAST(2 AS INT) AS [ZY - EXPRESS]
,CAST(3 AS INT) AS [OVERSEAS - DELUXE]
,CAST(4 AS INT) AS [OVERNIGHT J-FAST]
,CAST(5 AS INT) AS [CARGO TRANSPORT 5]
Then use like this:
SELECT h.*
FROM Sales.SalesOrderHeader h
JOIN ShipMethod.ShipMethodID const
ON h.ShipMethodID = const.[OVERNIGHT J-FAST]
Or like this:
SELECT h.*
FROM Sales.SalesOrderHeader h
WHERE h.ShipMethodID = (SELECT TOP 1 [OVERNIGHT J-FAST] FROM ShipMethod.ShipMethodID)
My workaround to missing constans is to give hints about the value to the optimizer.
DECLARE #Constant INT = 123;
SELECT *
FROM [some_relation]
WHERE [some_attribute] = #Constant
OPTION( OPTIMIZE FOR (#Constant = 123))
This tells the query compiler to treat the variable as if it was a constant when creating the execution plan. The down side is that you have to define the value twice.
No, but good old naming conventions should be used.
declare #MY_VALUE as int
There is no built-in support for constants in T-SQL. You could use SQLMenace's approach to simulate it (though you can never be sure whether someone else has overwritten the function to return something else…), or possibly write a table containing constants, as suggested over here. Perhaps write a trigger that rolls back any changes to the ConstantValue column?
Prior to using a SQL function run the following script to see the differences in performance:
IF OBJECT_ID('fnFalse') IS NOT NULL
DROP FUNCTION fnFalse
GO
IF OBJECT_ID('fnTrue') IS NOT NULL
DROP FUNCTION fnTrue
GO
CREATE FUNCTION fnTrue() RETURNS INT WITH SCHEMABINDING
AS
BEGIN
RETURN 1
END
GO
CREATE FUNCTION fnFalse() RETURNS INT WITH SCHEMABINDING
AS
BEGIN
RETURN ~ dbo.fnTrue()
END
GO
DECLARE #TimeStart DATETIME = GETDATE()
DECLARE #Count INT = 100000
WHILE #Count > 0 BEGIN
SET #Count -= 1
DECLARE #Value BIT
SELECT #Value = dbo.fnTrue()
IF #Value = 1
SELECT #Value = dbo.fnFalse()
END
DECLARE #TimeEnd DATETIME = GETDATE()
PRINT CAST(DATEDIFF(ms, #TimeStart, #TimeEnd) AS VARCHAR) + ' elapsed, using function'
GO
DECLARE #TimeStart DATETIME = GETDATE()
DECLARE #Count INT = 100000
DECLARE #FALSE AS BIT = 0
DECLARE #TRUE AS BIT = ~ #FALSE
WHILE #Count > 0 BEGIN
SET #Count -= 1
DECLARE #Value BIT
SELECT #Value = #TRUE
IF #Value = 1
SELECT #Value = #FALSE
END
DECLARE #TimeEnd DATETIME = GETDATE()
PRINT CAST(DATEDIFF(ms, #TimeStart, #TimeEnd) AS VARCHAR) + ' elapsed, using local variable'
GO
DECLARE #TimeStart DATETIME = GETDATE()
DECLARE #Count INT = 100000
WHILE #Count > 0 BEGIN
SET #Count -= 1
DECLARE #Value BIT
SELECT #Value = 1
IF #Value = 1
SELECT #Value = 0
END
DECLARE #TimeEnd DATETIME = GETDATE()
PRINT CAST(DATEDIFF(ms, #TimeStart, #TimeEnd) AS VARCHAR) + ' elapsed, using hard coded values'
GO
If you are interested in getting optimal execution plan for a value in the variable you can use a dynamic sql code. It makes the variable constant.
DECLARE #var varchar(100) = 'some text'
DECLARE #sql varchar(MAX)
SET #sql = 'SELECT * FROM table WHERE col = '''+#var+''''
EXEC (#sql)
For enums or simple constants, a view with a single row has great performance and compile time checking / dependency tracking ( cause its a column name )
See Jared Ko's blog post https://blogs.msdn.microsoft.com/sql_server_appendix_z/2013/09/16/sql-server-variables-parameters-or-literals-or-constants/
create the view
CREATE VIEW ShipMethods AS
SELECT CAST(1 AS INT) AS [XRQ - TRUCK GROUND]
,CAST(2 AS INT) AS [ZY - EXPRESS]
,CAST(3 AS INT) AS [OVERSEAS - DELUXE]
, CAST(4 AS INT) AS [OVERNIGHT J-FAST]
,CAST(5 AS INT) AS [CARGO TRANSPORT 5]
use the view
SELECT h.*
FROM Sales.SalesOrderHeader
WHERE ShipMethodID = ( select [OVERNIGHT J-FAST] from ShipMethods )
Okay, lets see
Constants are immutable values which are known at compile time and do not change for the life of the program
that means you can never have a constant in SQL Server
declare #myvalue as int
set #myvalue = 5
set #myvalue = 10--oops we just changed it
the value just changed
Since there is no build in support for constants, my solution is very simple.
Since this is not supported:
Declare Constant #supplement int = 240
SELECT price + #supplement
FROM what_does_it_cost
I would simply convert it to
SELECT price + 240/*CONSTANT:supplement*/
FROM what_does_it_cost
Obviously, this relies on the whole thing (the value without trailing space and the comment) to be unique. Changing it is possible with a global search and replace.
There are no such thing as "creating a constant" in database literature. Constants exist as they are and often called values. One can declare a variable and assign a value (constant) to it. From a scholastic view:
DECLARE #two INT
SET #two = 2
Here #two is a variable and 2 is a value/constant.
SQLServer 2022 (currently only as Preview available) is now able to Inline the function proposed by SQLMenace, this should prevent the performance hit described by some comments.
CREATE FUNCTION fnConstant() RETURNS INT AS BEGIN RETURN 2 END GO
SELECT is_inlineable FROM sys.sql_modules WHERE [object_id]=OBJECT_ID('dbo.fnConstant');
is_inlineable
1
SELECT dbo.fnConstant()
ExecutionPlan
To test if it also uses the value coming from the Function, I added a second function returning value "1"
CREATE FUNCTION fnConstant1()
RETURNS INT
AS
BEGIN
RETURN 1
END
GO
Create Temp Table with about 500k rows with Value 1 and 4 rows with Value 2:
DROP TABLE IF EXISTS #temp ;
create table #temp (value_int INT)
DECLARE #counter INT;
SET #counter = 0
WHILE #counter <= 500000
BEGIN
INSERT INTO #temp VALUES (1);
SET #counter = #counter +1
END
SET #counter = 0
WHILE #counter <= 3
BEGIN
INSERT INTO #temp VALUES (2);
SET #counter = #counter +1
END
create index i_temp on #temp (value_int);
Using the describe plan we can see that the Optimizer expects 500k values for
select * from #temp where value_int = dbo.fnConstant1(); --Returns 500001 rows
Constant 1
and 4 rows for
select * from #temp where value_int = dbo.fnConstant(); --Returns 4rows
Constant 2
Robert's performance test is interesting. And even in late 2022, the scalar functions are much slower (by an order of magnitude) than variables or literals. A view (as suggested mbobka) is somewhere in-between when used for this same test.
That said, using a loop like that in SQL Server is not something I'd ever do, because I'd normally be operating on a whole set.
In SQL 2019, if you use schema-bound functions in a set operation, the difference is much less noticeable.
I created and populated a test table:
create table #testTable (id int identity(1, 1) primary key, value tinyint);
And changed the test so that instead of looping and changing a variable, it queries the test table and returns true or false depending on the value in the test table, e.g.:
insert #testTable(value)
select case when value > 127
then #FALSE
else #TRUE
end
from #testTable with(nolock)
I tested 5 scenarios:
hard-coded values
local variables
scalar functions
a view
a table-valued function
running the test 10 times, yielded the following results:
scenario
min
max
avg
scalar functions
233
259
240
hard-coded values
236
265
243
local variables
235
278
245
table-valued function
243
272
253
view
244
267
254
Suggesting to me, that for set-based work in (at least) 2019 and better, there's not much in it.
set nocount on;
go
-- create test data table
drop table if exists #testTable;
create table #testTable (id int identity(1, 1) primary key, value tinyint);
-- populate test data
insert #testTable (value)
select top (1000000) convert(binary (1), newid())
from sys.all_objects a
, sys.all_objects b
go
-- scalar function for True
drop function if exists fnTrue;
go
create function dbo.fnTrue() returns bit with schemabinding as
begin
return 1
end
go
-- scalar function for False
drop function if exists fnFalse;
go
create function dbo.fnFalse () returns bit with schemabinding as
begin
return 0
end
go
-- table-valued function for booleans
drop function if exists dbo.tvfBoolean;
go
create function tvfBoolean() returns table with schemabinding as
return
select convert(bit, 1) as true, convert(bit, 0) as false
go
-- view for booleans
drop view if exists dbo.viewBoolean;
go
create view dbo.viewBoolean with schemabinding as
select convert(bit, 1) as true, convert(bit, 0) as false
go
-- create table for results
drop table if exists #testResults
create table #testResults (id int identity(1,1), test int, elapsed bigint, message varchar(1000));
-- define tests
declare #tests table(testNumber int, description nvarchar(100), sql nvarchar(max))
insert #tests values
(1, N'hard-coded values', N'
declare #testTable table (id int, value bit);
insert #testTable(id, value)
select id, case when t.value > 127
then 0
else 1
end
from #testTable t')
, (2, N'local variables', N'
declare #FALSE as bit = 0
declare #TRUE as bit = 1
declare #testTable table (id int, value bit);
insert #testTable(id, value)
select id, case when t.value > 127
then #FALSE
else #TRUE
end
from #testTable t'),
(3, N'scalar functions', N'
declare #testTable table (id int, value bit);
insert #testTable(id, value)
select id, case when t.value > 127
then dbo.fnFalse()
else dbo.fnTrue()
end
from #testTable t'),
(4, N'view', N'
declare #testTable table (id int, value bit);
insert #testTable(id, value)
select id, case when value > 127
then b.false
else b.true
end
from #testTable t with(nolock), viewBoolean b'),
(5, N'table-valued function', N'
declare #testTable table (id int, value bit);
insert #testTable(id, value)
select id, case when value > 127
then b.false
else b.true
end
from #testTable with(nolock), dbo.tvfBoolean() b')
;
declare #testNumber int, #description varchar(100), #sql nvarchar(max)
declare #testRuns int = 10;
-- execute tests
while #testRuns > 0 begin
set #testRuns -= 1
declare testCursor cursor for select testNumber, description, sql from #tests;
open testCursor
fetch next from testCursor into #testNumber, #description, #sql
while ##FETCH_STATUS = 0 begin
declare #TimeStart datetime2(7) = sysdatetime();
execute sp_executesql #sql;
declare #TimeEnd datetime2(7) = sysdatetime()
insert #testResults(test, elapsed, message)
select #testNumber, datediff_big(ms, #TimeStart, #TimeEnd), #description
fetch next from testCursor into #testNumber, #description, #sql
end
close testCursor
deallocate testCursor
end
-- display results
select test, message, count(*) runs, min(elapsed) as min, max(elapsed) as max, avg(elapsed) as avg
from #testResults
group by test, message
order by avg(elapsed);
The best answer is from SQLMenace according to the requirement if that is to create a temporary constant for use within scripts, i.e. across multiple GO statements/batches.
Just create the procedure in the tempdb then you have no impact on the target database.
One practical example of this is a database create script which writes a control value at the end of the script containing the logical schema version. At the top of the file are some comments with change history etc... But in practice most developers will forget to scroll down and update the schema version at the bottom of the file.
Using the above code allows a visible schema version constant to be defined at the top before the database script (copied from the generate scripts feature of SSMS) creates the database but used at the end. This is right in the face of the developer next to the change history and other comments, so they are very likely to update it.
For example:
use tempdb
go
create function dbo.MySchemaVersion()
returns int
as
begin
return 123
end
go
use master
go
-- Big long database create script with multiple batches...
print 'Creating database schema version ' + CAST(tempdb.dbo.MySchemaVersion() as NVARCHAR) + '...'
go
-- ...
go
-- ...
go
use MyDatabase
go
-- Update schema version with constant at end (not normally possible as GO puts
-- local #variables out of scope)
insert MyConfigTable values ('SchemaVersion', tempdb.dbo.MySchemaVersion())
go
-- Clean-up
use tempdb
drop function MySchemaVersion
go

Resources