Hi I need to create a view or stored procedure that combines data and returns a result set from 3 different databases on the same server using a column that holds a schema (db) name.
For Example on the first DB I have this table:
CREATE TABLE [dbo].[CloudUsers](
ID int IDENTITY(1,1) NOT NULL,
Username nvarchar(50) NULL,
MainDB nvarchar(100) NULL
) ON [PRIMARY]
Each CloudUser has a separate DB so next now I need to fetch the data from the User database using the MainDB name. The data I need is always 1 row cause I'm using aggregate functions / query.
So in the User MainDB let's say I have this table.
CREATE TABLE [dbo].[CLIENT](
ID int NOT NULL,
Name nvarchar(50) NULL,
ProjectDBName [nvarchar](100) NULL
CreationDate datetime NULL
) ON [PRIMARY]
And I query like:
select min(CreationDate) from MainDB.Client
The same Idea for the Client I need to fetch even more data from a 3rd database that points to the Client ProjectDBName. Again it's aggregate data:
select Count(id) as TotalTransactions from ProjectDBName.Journal
My final result should have records from all databases. It's readonly data that I need for statistics.
Final result set example:
CloudUsers.Username, MainDB->CreationDate, ProjectDBName->TotalTransaction
How can I achieve that ?
This is not easy - and without a schema and sample data, I can't give you a precise answer.
You need to iterate through each client, and use dynamic SQL to execute a the query against the mainDB and projectDB join. You can either do that in one gigantic "union" query, or by creating a temporary table and inserting the data into that temporary table, and then selecting from the temp table at the end of the query.
For you who are curious of how to solve this issue I have found my own solution using some cursors + dynamic and a simple table variable, enjoy.
ALTER PROCEDURE CloudAnalysis as
DECLARE #objcursor cursor
DECLARE #innercursor cursor
DECLARE #userid int
,#maindb nvarchar(100)
,#clientid int
,#name nvarchar(50)
,#projdb nvarchar(100)
,#stat nvarchar(50)
,#sql nvarchar(max)
,#vsql nvarchar(max)
,#rowcount int
DECLARE #result table(userid int,clientid int,maindb nvarchar(100),name nvarchar(50),projdb nvarchar(100),stat nvarchar(50))
SET #objcursor = CURSOR FORWARD_ONLY STATIC FOR SELECT c.id,c.maindb,u.client_id FROM dbo.ClientUsers c join dbo.UserClients u on c.id = u.user_id open #objcursor
FETCH NEXT FROM #objcursor INTO #userid,#maindb,#clientid
WHILE (##FETCH_STATUS=0)
BEGIN
IF (EXISTS (SELECT name
FROM master.dbo.sysdatabases
WHERE ('[' + name + ']' = #maindb
OR name = #maindb)))
BEGIN
set #sql = N'SELECT #name = c.name,#projdb=c.ProjectDBName FROM ' + #maindb + '.dbo.CLIENT c WHERE c.id = ' + cast(#clientid as nvarchar)
EXECUTE sp_executesql #sql, N'#name NVARCHAR(50) OUTPUT,#projdb NVARCHAR(100) OUTPUT',
#name = #name OUTPUT
,#projdb = #projdb OUTPUT
SELECT #rowcount = ##ROWCOUNT
IF #rowcount > 0
BEGIN
--print ' client: ' + cast(#clientid as nvarchar)+
--':' + #name + ' projdb: ' + #projdb
IF (EXISTS (SELECT name
FROM master.dbo.sysdatabases
WHERE ('[' + name + ']' = #projdb
OR name = #projdb)))
BEGIN
SET #sql = N'SELECT #stat = j.stat FROM ' + #projdb + '.dbo.JournalTransaction j'
EXECUTE sp_executesql #sql
,N'#stat NVARCHAR(50) OUTPUT'
,#stat = #stat OUTPUT
END
INSERT INTO #result (userid,clientid,maindb,name,projdb,stat)
VALUES (#userid,#clientid,#maindb,#name,#projdb,#stat)
END
END
FETCH NEXT FROM #objcursor INTO #userid,#maindb,#clientid
END
CLOSE #objcursor
DEALLOCATE #objcursor
SELECT * FROM #result
Related
I'm trying to build a stored procedure that will query multiple database depending on the databases required.
For example:
SP_Users takes a list of #DATABASES as parameters.
For each database it needs to run the same query and union the results together.
I believe a CTE could be my best bet so I have something like this at the moment.
SET #DATABASES = 'DB_1, DB_2' -- Two databases in a string listed
-- I have a split string function that will extract each database
SET #CURRENT_DB = 'DB_1'
WITH UsersCTE (Name, Email)
AS (SELECT Name, Email
FROM [#CURRENT_DB].[dbo].Users),
SELECT #DATABASE as DB, Name, Email
FROM UsersCTE
What I don't want to do is hard code the databases in the query. The steps I image are:
Split the parameter #DATABASES to extract and set the #CURRENT_DB Variable
Iterate through the query with a Recursive CTE until all the #DATABASES have been processed
Union all results together and return the data.
Not sure if this is the right approach to tackling this problem.
Using #databases:
As mentioned in the comments to your question, variables cant be used to dynamically select a database. Dynamic sql is indicated. You can start by building your template sql statement:
declare #sql nvarchar(max) =
'union all ' +
'select ''#db'' as db, name, email ' +
'from [#db].dbo.users ';
Since you have sql server 2016, you can split using the string_split function, with your #databases variable as input. This will result in a table with 'value' as the column name, which holds the database names.
Use the replace function to replace #db in the template with value. This will result in one sql statement for each database you passed into #databases. Then, concatenate the statements back together. Unfortunately, in version 2016, there's no built in function to do that. So we have to use the famous for xml trick to join the statements, then we use .value to convert it to a string, and finally we use stuff to get rid of the leading union all statement.
Take the results of the concatenated output, and overwrite the #sql variable. It is ready to go at this point, so execute it.
I do all that is described in this code:
declare #databases nvarchar(max) = 'db_1,db_2';
set #sql = stuff(
(
select replace(#sql, '#db', value)
from string_split(#databases, ',')
for xml path(''), type
).value('.[1]', 'nvarchar(max)')
, 1, 9, '');
exec(#sql);
Untested, of course, but if you print instead of execute, it seems to give the proper sql statement for your needs.
Using msForEachDB:
Now, if you didn't want to have to know which databases had 'users', such as if you're in an environment where you have a different database for every client, you can use sp_msForEachDb and check the structure first to make sure it has a 'users' table with 'name' and 'email' columns. If so, execute the appropriate statement. If not, execute a dummy statement. I won't describe this one, I'll just give the code:
declare #aggregator table (
db sysname,
name int,
email nvarchar(255)
);
insert #aggregator
exec sp_msforeachdb '
declare #sql nvarchar(max) = ''select db = '''''''', name = '''''''', email = '''''''' where 1 = 2'';
select #sql = ''select db = ''''?'''', name, email from ['' + table_catalog + ''].dbo.users''
from [?].information_schema.columns
where table_schema = ''dbo''
and table_name = ''users''
and column_name in (''name'', ''email'')
group by table_catalog
having count(*) = 2
exec (#sql);
';
select *
from #aggregator
I took the valid advice from others here and went with this which works great for what I need:
I decided to use a loop to build the query up. Hope this helps someone else looking to do something similar.
CREATE PROCEDURE [dbo].[SP_Users](
#DATABASES VARCHAR(MAX) = NULL,
#PARAM1 VARCHAR(250),
#PARAM2 VARCHAR(250)
)
BEGIN
SET NOCOUNT ON;
--Local variables
DECLARE
#COUNTER INT = 0,
#SQL NVARCHAR(MAX) = '',
#CURRENTDB VARCHAR(50) = NULL,
#MAX INT = 0,
#ERRORMSG VARCHAR(MAX)
--Check we have databases entered
IF #DATABASES IS NULL
BEGIN
RAISERROR('ERROR: No Databases Provided,
Please Provide a list of databases to execute procedure. See stored procedure:
[SP_Users]', 16, 1)
RETURN
END
-- SET Number of iterations based on number of returned databases
SET #MAX = (SELECT COUNT(*) FROM
(SELECT ROW_NUMBER() OVER (ORDER BY i.value) AS RowNumber, i.value
FROM dbo.udf_SplitVariable(#DATABASES, ',') AS i)X)
-- Build SQL Statement
WHILE #COUNTER < #MAX
BEGIN
--Set the current database
SET #CURRENTDB = (SELECT X.Value FROM
(SELECT ROW_NUMBER() OVER (ORDER BY i.value) AS RowNumber, i.value
FROM dbo.udf_SplitVariable(#DATABASES, ',') AS i
ORDER BY RowNumber OFFSET #COUNTER
ROWS FETCH NEXT 1 ROWS ONLY) X);
SET #SQL = #SQL + N'
(
SELECT Name, Email
FROM [' + #CURRENTDB + '].[dbo].Users
WHERE
(Name = #PARAM1 OR #PARAM1 IS NULL)
(Email = #PARAM2 OR #PARAM2 IS NULL)
) '
+ N' UNION ALL '
END
PRINT #CURRENTDB
PRINT #SQL
SET #COUNTER = #COUNTER + 1
END
-- remove last N' UNION ALL '
IF LEN(#SQL) > 11
SET #SQL = LEFT(#SQL, LEN(#SQL) - 11)
EXEC sp_executesql #SQL, N'#CURRENTDB VARCHAR(50),
#PARAM1 VARCHAR(250),
#PARAM2 VARCHAR(250)',
#CURRENTDB,
#PARAM1 ,
#PARAM2
END
Split Variable Function
CREATE FUNCTION [dbo].[udf_SplitVariable]
(
#List varchar(8000),
#SplitOn varchar(5) = ','
)
RETURNS #RtnValue TABLE
(
Id INT IDENTITY(1,1),
Value VARCHAR(8000)
)
AS
BEGIN
--Account for ticks
SET #List = (REPLACE(#List, '''', ''))
--Account for 'emptynull'
IF LTRIM(RTRIM(#List)) = 'emptynull'
BEGIN
SET #List = ''
END
--Loop through all of the items in the string and add records for each item
WHILE (CHARINDEX(#SplitOn,#List)>0)
BEGIN
INSERT INTO #RtnValue (value)
SELECT Value = LTRIM(RTRIM(SUBSTRING(#List, 1, CHARINDEX(#SplitOn, #List)-1)))
SET #List = SUBSTRING(#List, CHARINDEX(#SplitOn,#List) + LEN(#SplitOn), LEN(#List))
END
INSERT INTO #RtnValue (Value)
SELECT Value = LTRIM(RTRIM(#List))
RETURN
END
The question is quite extensive, please bear with me. I have a single mapping table with the following structure:
This particular table is used in the process of generating a hierarchy. The order and position of the columns in the table indicate the order of hierarchy (Organization, Category, Continent, Country.. etc.) Each entity in this hierarchy has a related table with associated Id and Name. For example, there is a Country table with CountryId and CountryName. Note that since the MappingTable's values are all nullable there are no foreign key constraints.
I want to generate a procedure that will do the following:
Based on conditions provided, retrieve values of the next entity in the hierarchy. For example, if the OrganizationId and CategoryId are given, the values of ContinentId that satisfy said condition need to be retrieved.
Also, if the value of ContinentId is NULL, then the values of CountryId need to be retrieved. Here, given the condition OrganizationId = 1 and CategoryId = 1 the procedure should return the list of RegionId.
In addition to retrieving the RegionId, the corresponding RegionName should be retrieved from the Region Table.
So far, the procedure looks something like this - just a few things to explain here.
ALTER PROCEDURE [dbo].[GetHierarchy]
(
#MappingTableName VARCHAR(30),
#Position VARCHAR(5),
-- Given in the form of Key-value pairs 'OrganizationId:1,CategoryId:1'
#InputData VARCHAR(MAX),
#Separator CHAR(1),
#KeyValueSeperator CHAR(1)
)
AS
BEGIN
DECLARE #Sql NVARCHAR(MAX)
DECLARE #Result NVARCHAR(MAX)
DECLARE #Sql1 NVARCHAR(MAX);
DECLARE #TableName NVARCHAR(30)
DECLARE #Exists bit
SELECT #TableName = COLUMN_NAME from INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = #MappingTableName AND ORDINAL_POSITION = #position
SET #TableName = SUBSTRING(#TableName,0,LEN(#TableName) - 1)
-- Returns a dynamic query like "SELECT ContinentId from Continent WHERE OrganizationId = 1 and CategoryId = 1".
SELECT #Sql = [dbo].[KeyValuePairs](#TableName, #InputData, #Separator, #KeyValueSeperator)
SET #Sql1 = N'SET #Exists = CASE WHEN EXISTS(' + #Sql + N' AND ' + #TableName + N'Id IS NOT NULL) THEN 1 ELSE 0 END'
PRINT #Sql
EXEC sp_executesql #Sql1,
N'#Exists bit OUTPUT',
#Exists = #Exists OUTPUT
IF(#Exists = 1)
BEGIN
SET #Sql1 = 'SELECT ' + #TableName + 'Id, ' + #TableName + 'Name FROM '+ #TableName+' WHERE ' + #TableName +'Id IN (' + #Sql + ')';
PRINT #Sql1
EXECUTE sp_executesql #SQL1
END
ELSE
BEGIN
--PRINT 'NOT EXISTS'
DECLARE #nextPosition INT
SELECT #nextPosition = CAST(#position AS INT)
SET #nextPosition = #nextPosition + 1
SET #Position = CONVERT(VARCHAR(5), CAST(#position AS INT))
EXEC [dbo].[GetHierarchy] #MappingTableName, #Position, #InputData, #Separator, #KeyValueSeperator
END
END
The logic of this procedure is such that, I get the name of the column at a particular position (based on the conditions here, it is Continent) and generate the dynamic query to retrieve the next column's values based on the condition of the input condition (I am using a separate function to do this for me).
Once retrieved, I run the query to check if it returns any rows. If the query returns rows, then I retrieve the corresponding ContinentName from the Continent table. If no rows are returns, I recursively call the procedure again with the next position as the input.
On the business side of things, it seems like a two step process. But, as a procedure it is quite complex, extensive and - not to mention, recursive. Is there an easier way to do this? I am not familiar with CTEs - can the same logic be implemented using CTEs?
This is quite similar to what is asked here: Working with a dynamic hierarchy SQL Server
Might be the little lengthy approach. Try this
DECLARE #T TABLE
(
SeqNo INT IDENTITY(1,1),
CatId INT,
Country INT,
StateId INT,
DistId INT
)
DECLARE #State TABLE
(
StateId INT,
StateNm VARCHAR(20)
)
DECLARE #Country TABLE
(
CountryId INT,
CountryNm VARCHAR(20)
)
INSERT INTO #State
VALUES(3,'FL')
INSERT INTO #Country
VALUES(2,'USA')
INSERT INTO #T(CatId)
VALUES(1)
INSERT INTO #T(CatId,Country)
VALUES(1,2)
INSERT INTO #T(CatId,StateId)
VALUES(1,3)
;WITH CTE
AS
(
SELECT
*,
IdVal = COALESCE(Country,StateId,DistId),
IdCol = COALESCE('Country '+CAST(Country AS VARCHAR(50)),'StateId '+CAST(StateId AS VARCHAR(50)),'DistId '+CAST(DistId AS VARCHAR(50)))
FROM #T
WHERE CatId = 1
),C2
AS
(
SELECT
SeqNo,
CatId,
Country,
StateId,
DistId,
IdVal,
IdCol = LTRIM(RTRIM(SUBSTRING(IdCol,1,CHARINDEX(' ',IdCol))))
FROM CTE
)
SELECT
C2.SeqNo,
C2.CatId,
S.StateNm,
C.CountryNm
FROM C2
LEFT JOIN #State S
ON C2.IdCol ='StateId'
AND C2.IdVal = S.StateId
LEFT JOIN #Country C
ON C2.IdCol ='Country '
AND C2.IdVal = c.CountryId
I have a form with 3 check box dropdown lists enabling multiple selection from each control.
Lets say for talking sake its an accommodation table I am querying and the check box dropdown lists are 'AccommodationName', 'Company', and 'Nights'.
So potentially I could be passing in multiple values from each control and I want to return an aggregated query relevant to all data input.
How should I be going about this query?
Is the query going to have to be dynamic sql?
Please note, I am using sql server 2005.
You will need to create a split function inside you database,
Definition Of Split Function
CREATE FUNCTION [dbo].[split]
(
#delimited NVARCHAR(MAX),
#delimiter NVARCHAR(100)
)
RETURNS #t TABLE (id INT IDENTITY(1,1), val NVARCHAR(MAX))
AS
BEGIN
DECLARE #xml XML
SET #xml = N'<t>' + REPLACE(#delimited,#delimiter,'</t><t>') + '</t>'
INSERT INTO #t(val)
SELECT r.value('.','varchar(MAX)') as item
FROM #xml.nodes('/t') as records(r)
RETURN
END
Stored Procedure
Then you need to create a stored procedure which will build sql query dynamically and use this split function to handle multiple values passed as a comma deliminated list.
CREATE PROCEDURE GetData
#AccommodationName VARCHAR(1000) = NULL,
#Company VARCHAR(1000) = NULL,
#Nights VARCHAR(1000) = NULL
AS
BEGIN
SET NOCOUNT ON;
DECLARE #SQL NVARCHAR(MAX);
SET #SQL = N' SELECT * FROM TableName WHERE 1 = 1 '
+ CASE WHEN #AccommodationName IS NOT NULL
THEN N' AND AccommodationName IN (SELECT Val FROM dbo.split(#AccommodationName )) '
ELSE N'' END
+ CASE WHEN #Company IS NOT NULL
THEN N' AND Company IN (SELECT Val FROM dbo.split(#Company)) '
ELSE N'' END
+ CASE WHEN #Nights IS NOT NULL
THEN N' AND Nights IN (SELECT Val FROM dbo.split(#Nights)) '
ELSE N'' END
EXECUTE sp_executesql #SQL
,N'#AccommodationName VARCHAR(1000), #Company VARCHAR(1000), #Nights VARCHAR(1000)'
,#AccommodationName
,#Company
,#Nights
END
I have some tables for storing different file information, like thumbs, images, datasheets, ...
I'm writing a stored procedure to retrieve filename of a specific ID. something like:
CREATE PROCEDURE get_file_name(
#id int,
#table nvarchar(50)
)as
if #table='images'
select [filename] from images
where id = #id
if #table='icons'
select [filename] from icons
where id = #id
....
How can I rewrite this procedure using case when statement or should I just use table name as variable?
You can't use case .. when to switch between a table in the FROM clause (like you can in a conditional ORDER BY). i.e. so the following:
select * from
case when 1=1
then t1
else t2
end;
won't work.
So you'll need to use dynamic SQL. It's best to parameterize the query as far as possible, for example the #id value can be parameterized:
-- Validate #table is E ['images', 'icons', ... other valid names here]
DECLARE #sql NVARCHAR(MAX)
SET #sql = 'select [filename] from **TABLE** where id = #id';
SET #sql = REPLACE(#sql, '**TABLE**', #table);
sp_executesql #sql, N'#id INT', #id = #id;
As with all dynamic Sql, note that unparameterized values which are substituted into the query (like #table), make the query vulnerable to Sql Injection attacks. As a result, I would suggest that you ensure that #table comes from a trusted source, or better still, the value of #table is compared to a white list of permissable tables prior to execution of the query.
Just build SQL string in another variable and EXECUTE it
DECLARE #sql AS NCHAR(500)
SET #sql=
'SELECT [filename] '+
' FROM '+#table+
' WHERE id = #id'
EXECUTE(#sql)
CREATE PROCEDURE get_file_name(
#id int,
#table nvarchar(50)
)as
DECLARE #SQL nvarchar(max);
SET #SQL = 'select [filename] from ' + #table + ' where id = ' + #id
EXECUTE (#SQL)
The code is as follows:
ALTER PROCEDURE dbo.pdpd_DynamicCall
#SQLString varchar(4096) = null
AS
Begin
create TABLE #T1 ( column_1 varchar(10) , column_2 varchar(100) )
insert into #T1
execute ('execute ' + #SQLString )
select * from #T1
End
The problem is that I want to call different procedures that can give back different columns.
Therefore I would have to define the table #T1 generically.
But I don't know how.
Can anyone help me on this problem?
Try:
SELECT into #T1 execute ('execute ' + #SQLString )
And this smells real bad like an sql injection vulnerability.
correction (per #CarpeDiem's comment):
INSERT into #T1 execute ('execute ' + #SQLString )
also, omit the 'execute' if the sql string is something other than a procedure
You can define a table dynamically just as you are inserting into it dynamically, but the problem is with the scope of temp tables. For example, this code:
DECLARE #sql varchar(max)
SET #sql = 'CREATE TABLE #T1 (Col1 varchar(20))'
EXEC(#sql)
INSERT INTO #T1 (Col1) VALUES ('This will not work.')
SELECT * FROM #T1
will return with the error "Invalid object name '#T1'." This is because the temp table #T1 is created at a "lower level" than the block of executing code. In order to fix, use a global temp table:
DECLARE #sql varchar(max)
SET #sql = 'CREATE TABLE ##T1 (Col1 varchar(20))'
EXEC(#sql)
INSERT INTO ##T1 (Col1) VALUES ('This will work.')
SELECT * FROM ##T1
Hope this helps,
Jesse
Be careful of a global temp table solution as this may fail if two users use the same routine at the same time as a global temp table can be seen by all users...
create a global temp table with a GUID in the name dynamically. Then you can work with it in your code, via dyn sql, without worry that another process calling same sproc will use it. This is useful when you dont know what to expect from the underlying selected table each time it runs so you cannot created a temp table explicitly beforehand. ie - you need to use SELECT * INTO syntax
DECLARE #TmpGlobalTable varchar(255) = 'SomeText_' + convert(varchar(36),NEWID())
-- select #TmpGlobalTable
-- build query
SET #Sql =
'SELECT * INTO [##' + #TmpGlobalTable + '] FROM SomeTable'
EXEC (#Sql)
EXEC ('SELECT * FROM [##' + #TmpGlobalTable + '] ')
EXEC ('DROP TABLE [##' + #TmpGlobalTable + ']')
PRINT 'Dropped Table ' + #TmpGlobalTable
INSERT INTO #TempTable
EXEC(#SelectStatement)
Try Below code for creating temp table dynamically from Stored Procedure Output using T-SQL
declare #ExecutionName varchar(1000) = 'exec [spname] param1,param2 '
declare #sqlStr varchar(max) = ''
declare #tempTableDef nvarchar(max) =
(
SELECT distinct
STUFF(
(
SELECT ','+a.[name]+' '+[system_type_name]
+'
' AS [text()]
FROM sys.dm_exec_describe_first_result_set (#ExecutionName, null, 0) a
ORDER BY a.column_ordinal
FOR XML PATH ('')
), 1, 1, '') tempTableDef
FROM sys.dm_exec_describe_first_result_set (#ExecutionName, null, 0) b
)
IF ISNULL(#tempTableDef ,'') = '' RAISERROR( 'Invalid SP Configuration. At least one column is required in Select list of SP output.',16,1) ;
set #tempTableDef='CREATE TABLE #ResultDef
(
' + REPLACE(#tempTableDef,'
','') +'
)
INSERT INTO #ResultDef
' + #ExecutionName
Select #sqlStr = #tempTableDef +' Select * from #ResultDef '
exec(#sqlStr)
DECLARE #EmpGroup INT =3 ,
#IsActive BIT=1
DECLARE #tblEmpMaster AS TABLE
(EmpCode VARCHAR(20),EmpName VARCHAR(50),EmpAddress VARCHAR(500))
INSERT INTO #tblEmpMaster EXECUTE SPGetEmpList #EmpGroup,#IsActive
SELECT * FROM #tblEmpMaster
CREATE PROCEDURE dbo.pdpd_DynamicCall
AS
DECLARE #SQLString_2 NVARCHAR(4000)
SET NOCOUNT ON
Begin
--- Create global temp table
CREATE TABLE ##T1 ( column_1 varchar(10) , column_2 varchar(100) )
SELECT #SQLString_2 = 'INSERT INTO ##T1( column_1, column_2) SELECT column_1 = "123", column_2 = "MUHAMMAD IMRON"'
SELECT #SQLString_2 = REPLACE(#SQLString_2, '"', '''')
EXEC SP_EXECUTESQL #SQLString_2
--- Test Display records
SELECT * FROM ##T1
--- Drop global temp table
IF OBJECT_ID('tempdb..##T1','u') IS NOT NULL
DROP TABLE ##T1
End
Not sure if I understand well, but maybe you could form the CREATE statement inside a string, then execute that String? That way you could add as many columns as you want.