I've read multiple threads on this topic, and there does seem to be a way to do this dynamically, but I'm getting a syntax/compile error on the code.
I am trying to dynamically pull multiple rows for a temp table that simply has one column. The definition of the temp table is one field called acct_name. The challenge seems to be the "dynamics" of the fact that there will be over 50+ rows in the temp table, which is subject to change at any time.
I followed a previous poster's example, but am still getting compile/runtime errors.
I'm a rather novice SQL person, so you'll have to excuse the crudity of my code or my question.
DECLARE #idList varchar(500)
SELECT #idList = COALESCE(#idList + ',', '') + acct_name
FROM ##tmp_accts
DECLARE #sqlToRun varchar(1000)
SET #sqlToRun = '
SELECT *
FROM (
SELECT acct_name FROM ##tmp_accts
) AS src
PIVOT (
MAX(acct_name) FOR acct_name IN ('+ #idList +')
) AS pvt'
EXEC (#sqlToRun)
Does anyone have an obvious suggestion, I think it's very close to working.....
FOR EXAMPLE,
Let's say for sake of example we have the following acct_names - '12345','23456','34567','45678'.
The desire result is to return one row with 4 columns each with the respective value of acct_name. HOWEVER, the acct name is dynamic and is not known in advance, nor is the count of acct_name known in advance. A temp table is generated on the fly which determines all of the relevant acct_names for that particular run. It will vary with each run, each day that the query is run.
Thank you.....
Thru an article available thru Microsoft, the following solution does the job apporpriately.....
DECLARE
#columns NVARCHAR(MAX) = '',
#sql NVARCHAR(MAX) = '';
-- select the category names
SELECT
#columns+=QUOTENAME(acct_name) + ','
FROM
##tmp_accts
GROUP BY
acct_name;
-- remove the last comma
SET #columns = LEFT(#columns, LEN(#columns) - 1);
-- construct dynamic SQL
SET #sql ='
SELECT * FROM
(
SELECT DISTINCT acct_name
FROM
##tmp_accts
) t
PIVOT(
COUNT(acct_name)
FOR acct_name IN ('+ #columns + ')
) AS piv;';
-- execute the dynamic SQL
EXECUTE sp_executesql #sql;
Related
I am working on a dynamic SQL query for a MsSQL stored procedure.
There is a table search.ProfileFields that contains the actual column names in a table I need to query.
My goal is to have the SQL select the specific column in the table, dynamically from its parent query..
A little confusing, heres an example:
DECLARE #sql nvarchar(max)
SELECT #sql = '
SELECT
pfs.SectionID,
pfs.SectionName,
pfs.Link,
(
SELECT
pf.FieldID,
pf.FieldTitle,
pf.FieldSQL,
pf.Restricted,
pf.Optional,
(
SELECT
pf.FieldSQL
FROM
Resources.emp.EmployeeComplete as e
WHERE
e.QID = #QID
) as Value
FROM
search.ProfileFields as pf
WHERE
pf.SectionID = pfs.SectionID
ORDER BY
pf.[Order]
FOR XML PATH (''field''), ELEMENTS, TYPE, ROOT (''fields'')
)
FROM
search.ProfileFieldSections as pfs
WHERE
pfs.Status = 1
FOR XML PATH (''data''), ELEMENTS, TYPE, ROOT (''root'')'
PRINT #sql
EXECUTE sp_executesql #sql, N'#QID varchar(10)', #QID = #QID
In the inner most select. I am querying pf.FieldSQL. I am looking for the actual value that was received by the parent select.
search.ProfileFields has a column called FieldSQL with a few results such as Name, Age, Location.
That is what I am trying to get my inner most select to do.
SELECT Name FROM ... - Name in this case comes from the value of pf.FieldSQL.
How can I go about querying a dynamic column name in this situation?
Have a look at this answer for a couple of suggestions. If your table definition is complex or changes occasionally you probably should use pivot. Here's one that might work for you, so long as column names in the FieldSQL column are well defined, there are not too many of them, and they don't ever change or get added to:
DECLARE #sql nvarchar(max)
SELECT #sql = '
SELECT
pfs.SectionID,
pfs.SectionName,
pfs.Link,
(
SELECT
pf.FieldID,
pf.FieldTitle,
pf.FieldSQL,
pf.Restricted,
pf.Optional,
(
SELECT case pf.FieldSQL
when 'Name' then e.Name
when 'DOB' then convert(nvarchar(10), e.DOB, 126)
-- ... etc.
-- NOTE: may need to aggregated depending on uniqueness of QID:
-- when 'Name' then min(e.Name)
-- when 'DOB' then convert(nvarchar(10), min(e.DOB), 126)
end
FROM
Resources.emp.EmployeeComplete as e
WHERE
e.QID = #QID
) as Value
FROM
search.ProfileFields as pf
WHERE
pf.SectionID = pfs.SectionID
ORDER BY
pf.[Order]
FOR XML PATH (''field''), ELEMENTS, TYPE, ROOT (''fields'')
)
FROM
search.ProfileFieldSections as pfs
WHERE
pfs.Status = 1
FOR XML PATH (''data''), ELEMENTS, TYPE, ROOT (''root'')'
PRINT #sql
EXECUTE sp_executesql #sql, N'#QID varchar(10)', #QID = #QID
Take a look at "PIVOT" operators here PIVOT
Thisshould give you some ideas how to use them.
I received help previously on creating a query for listing total sales by month for a list of jobs. However, I cannot put this into a view as a variable needs declared.
I thought I could use a Table Valued Function (which I've never used before) but apparently I need to define the columns of the table which I cannot do if they are to be dynamic (the columns are yyyy-mm).
Can someone suggest how I could approach this problem please?
I am not familiar with creating anything other than views so this is completely new to me. Obviously if using the TVF isn't the best method please advise.
The query:
Declare #SQL varchar(max) = Stuff((Select Distinct ',' + QuoteName(YearMonth) From v_JobSalesByMonth Order by 1 For XML Path('')),1,1,'')
Select #SQL = '
Select [JobID], [TotalJobSales],' + #SQL + '
From (
Select JobID
,TotalJobSales = sum(SalesForMonth) over (Partition By JobID)
,YearMonth
,SalesForMonth
From v_JobSalesByMonth A) A
Pivot (sum(SalesForMonth) For [YearMonth] in (' + #SQL + ') ) p'
Exec(#SQL);
I started to create the function but am confused about how to define the dynamic columns.
CREATE FUNCTION db.tvfnJobSalesByMonthPivot (#JobID INT)
RETURNS #JobSalesByMonth TABLE
(
JobID int PRIMARY KEY NOT NULL,
TotalJobSales decimal(6,2) NULL,
2016-12 (ermmmmm?)
)
Many thanks in advance.
Paul
I'm trying to merge a very wide table from a source (linked Oracle server) to a target table (SQL Server 2012) w/o listing all the columns. Both tables are identical except for the records in them.
This is what I have been using:
TRUNCATE TABLE TargetTable
INSERT INTO TargetTable
SELECT *
FROM SourceTable
When/if I get this working I would like to make it a procedure so that I can pass into it the source, target and match key(s) needed to make the update. For now I would just love to get it to work at all.
USE ThisDatabase
GO
DECLARE
#Columns VARCHAR(4000) = (
SELECT COLUMN_NAME + ','
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'TargetTable'
FOR XML PATH('')
)
MERGE TargetTable AS T
USING (SELECT * FROM SourceTable) AS S
ON (T.ID = S.ID AND T.ROWVERSION = S.ROWVERSION)
WHEN MATCHED THEN
UPDATE SET #Columns = S.#Columns
WHEN NOT MATCHED THEN
INSERT (#Columns)
VALUES (S.#Columns)
Please excuse my noob-ness. I feel like I'm only half way there, but I don't understand some parts of SQL well enough to put it all together. Many thanks.
As previously mentioned in the answers, if you don't want to specify the columns , then you have to write a dynamic query.
Something like this in your case should help:
DECLARE
#Columns VARCHAR(4000) = (
SELECT COLUMN_NAME + ','
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'TargetTable'
FOR XML PATH('')
)
DECLARE #MergeQuery NVARCHAR(MAX)
DECLARE #UpdateQuery VARCHAR(MAX)
DECLARE #InsertQuery VARCHAR(MAX)
DECLARE #InsertQueryValues VARCHAR(MAX)
DECLARE #Col VARCHAR(200)
SET #UpdateQuery='Update Set '
SET #InsertQuery='Insert ('
SET #InsertQueryValues=' Values('
WHILE LEN(#Columns) > 0
BEGIN
SET #Col=left(#Columns, charindex(',', #Columns+',')-1);
IF #Col<> 'ID' AND #Col <> 'ROWVERSION'
BEGIN
SET #UpdateQuery= #UpdateQuery+ 'TargetTable.'+ #Col + ' = SourceTable.'+ #Col+ ','
SET #InsertQuery= #InsertQuery+#Col + ','
SET #InsertQueryValues=#InsertQueryValues+'SourceTable.'+ #Col+ ','
END
SET #Columns = stuff(#Columns, 1, charindex(',', #Columns+','), '')
END
SET #UpdateQuery=LEFT(#UpdateQuery, LEN(#UpdateQuery) - 1)
SET #InsertQuery=LEFT(#InsertQuery, LEN(#InsertQuery) - 1)
SET #InsertQueryValues=LEFT(#InsertQueryValues, LEN(#InsertQueryValues) - 1)
SET #InsertQuery=#InsertQuery+ ')'+ #InsertQueryValues +')'
SET #MergeQuery=
N'MERGE TargetTable
USING SourceTable
ON TargetTable.ID = SourceTable.ID AND TargetTable.ROWVERSION = SourceTable.ROWVERSION ' +
'WHEN MATCHED THEN ' + #UpdateQuery +
' WHEN NOT MATCHED THEN '+#InsertQuery +';'
Execute sp_executesql #MergeQuery
If you want more information about Merge, you could read the this excellent article
Don't feel bad. It takes time. Merge has interesting syntax. I've actually never used it. I read Microsoft's documentation on it, which is very helpful and even has examples. I think I covered everything. I think there may be a slight amount of tweaking you might have to do, but I think it should work.
Here's the documentation for MERGE:
https://msdn.microsoft.com/en-us/library/bb510625.aspx
As for your code, I commented pretty much everything to explain it and show you how to do it.
This part is to help write your merge statement
USE ThisDatabase --This says what datbase context to use.
--Pretty much what database your querying.
--Like this: database.schema.objectName
GO
DECLARE
#SetColumns VARCHAR(4000) = (
SELECT CONCAT(QUOTENAME(COLUMN_NAME),' = S.',QUOTENAME(COLUMN_NAME),',',CHAR(10)) --Concat just says concatenate these values. It's adds the strings together.
--QUOTENAME adds brackets around the column names
--CHAR(10) is a line break for formatting purposes(totally optional)
FROM INFORMATION_SCHEMA.COLUMNS
--WHERE TABLE_NAME = 'TargetTable'
FOR XML PATH('')
) --This uses some fancy XML trick to get your Columns concatenated into one row.
--What really is in your table is a column of your column names in different rows.
--BTW If the columns names in both tables are identical, then this will work.
DECLARE #Columns VARCHAR(4000) = (
SELECT QUOTENAME(COLUMN_NAME) + ','
FROM INFORMATION_SCHEMA.COLUMNS
--WHERE TABLE_NAME = 'TargetTable'
FOR XML PATH('')
)
SET #Columns = SUBSTRING(#Columns,0,LEN(#Columns)) -- this gets rid off the comma at the end of your list
SET #SetColumns = SUBSTRING(#SetColumns,0,LEN(#SetColumns)) --same thing here
SELECT #SetColumns --Your going to want to copy and paste this into your WHEN MATCHED statement
SELECT #Columns --Your going to want to copy this into your WHEN NOT MATCHED statement
GO
Merge Statement
Especially look at my notes on ROWVERSION.
MERGE INTO TargetTable AS T
USING SourceTable AS S --Don't really need to write SELECT * FROM since you need the whole table anyway
ON (T.ID = S.ID AND T.[ROWVERSION] = S.[ROWVERSION]) --These are your matching parameters
--One note on this, if ROWVERSION is different versions of the same data you don't want to have RowVersion here
--Like lets say you have ID 1 ROWVERSION 2 in your source but only version 1 in your targetTable
--If you leave T.ID =S.ID AND T.ROWVERSION = S.ROWVERSION, then it will insert the new ROWVERSION
--So you'll have two versions of ID 1
WHEN MATCHED THEN --When TargetTable ID and ROWVERSION match in the matching parameters
--Update the values in the TargetTable
UPDATE SET /*Copy and Paste #SetColumnss here*/
--Should look like this(minus the "--"):
--Col1 = S.Col1,
--Col2 = S.Col2,
--Col3 = S.Col3,
--Etc...
WHEN NOT MATCHED THEN --This says okay there are no rows with the existing ID, now insert a new row
INSERT (col1,col2,col3) --Copy and paste #Columns in between the parentheses. Should look like I show it. Note: This is insert into target table so your listing the target table columns
VALUES (col1,col2,col3) --Same thing here. This is the list of source table columns
I have a SourceTable and a table variable #TQueries containing various T-SQL predicates that target SourceTable.
The expected result is to dynamically generate SELECT statements that return a list of Id's as specified by the predicates in #TQueries. Each dynamically generated SELECT statement also needs to execute in a particular order, and the final set of values needs to be unique and the ordering must be preserved.
Fortunately, there's a limit to how many values need to be retrieved and how many dynamic queries need to be generated. The Id list should contain at most 10 Ids, and we don't expect more than 7 queries.
The following is a sample of this setup, not the actual data/database:
-- Set up some test data, this is quick and dirty just to provide some data to test against
IF NOT EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[SourceTable]') AND type in (N'U'))
BEGIN
-- Create a numbers table, sorta
SELECT TOP 20
IDENTITY(INT,1,1) AS Id,
ABS(CHECKSUM(NewId())) % 100 AS [SomeValue]
INTO [SourceTable]
FROM sysobjects a
END
DECLARE #TQueries TABLE (
[Ordinal] INT,
[WherePredicate] NVARCHAR(MAX),
[OrderByPredicate] NVARCHAR(MAX)
);
-- Simulate SELECTs with different order by that get different data due to varying WHERE clauses and ORDER conditions
INSERT INTO #TQueries VALUES ( 1, N'[Id] IN (6,11,13,7,10,3,15)', '[SomeValue] ASC' ) -- Sort Asc
INSERT INTO #TQueries VALUES ( 2, N'[Id] IN (9,15,14,20,17)', '[SomeValue] DESC' ) -- Sort Desc
INSERT INTO #TQueries VALUES ( 3, N'[Id] IN (20,10,1,16,11,19,9,15,17,6,2,3,13)', 'NEWID()' ) -- Sort Random
My main issue has been avoiding the use of a CURSOR or iterating through the rows one by one. The closest I've come to a set operation that meets this criteria is using a table variable to store the results of each query or a massive CTE.
Suggestions and comments are welcome.
Here's a solution that builds a single statement both to run all the queries and to return the results.
It uses a similar approach as in your answer when iterating over the #TQueries table, i.e. it also uses {...} tokens where column values from #TQuery should go, and it puts the values there with nested REPLACE() calls.
Other than that, it heavily depends on ranking functions, and I'm not sure if doesn't really abuse them. You'd need to test this method before deciding if it's better or worse than the one you've got so far.
DECLARE #QueryTemplate nvarchar(max), #FinalSQL nvarchar(max);
SET #QueryTemplate =
N'SELECT
[Id],
QueryRank = {Ordinal},
RowRank = ROW_NUMBER() OVER (ORDER BY {OrderByPredicate})
FROM [dbo].[SourceTable]
WHERE {WherePredicate}
';
SET #FinalSQL =
N'WITH AllData AS (
' +
SUBSTRING(
(
SELECT
'UNION ALL ' +
REPLACE(REPLACE(REPLACE(#QueryTemplate,
'{Ordinal}' , [Ordinal] ),
'{OrderByPredicate}', [OrderByPredicate]),
'{WherePredicate}' , [WherePredicate] )
FROM #TQueries
ORDER BY [Ordinal]
FOR XML PATH (''), TYPE
).value('.', 'nvarchar(max)'),
11, -- starting just after the first 'UNION ALL '
CAST(0x7FFFFFFF AS int) -- max int; no need to specify the exact length
) +
'),
RankedData AS (
SELECT
[Id],
QueryRank,
RowRank,
ValueRank = ROW_NUMBER() OVER (PARTITION BY [Id] ORDER BY QueryRank)
FROM AllData
)SELECT TOP (#top)
[Id]
FROM RankedData
WHERE ValueRank = 1
ORDER BY
QueryRank,
RowRank
';
PRINT #FinalSQL;
EXECUTE sp_executesql #FinalSQL, N'#top int', 10;
Basically, every subquery gets these auxiliary columns:
QueryRank – a constant value (within the subquery's result set) derived from [Ordinal];
RowRank – a ranking assigned to a row based on the [OrderByPredicate].
The result sets are UNIONed and then every entry of every unique value is again ranked (ValueRank) based on the query ranking.
When pulling the final result set, duplicates are suppressed (by the condition ValueRank = 1), and QueryRank and RowRank are used in the ORDER BY clause to preserve the original row order.
I used EXECUTE sp_executesql #query instead of EXECUTE (#query), because the former allows you to add parameters to the query. In particular, I parametrised the number of results to return (the argument of TOP). But you could certainly concatenate that value into the dynamic script directly, just like other things, if you prefer EXECUTE () over EXECUTE sq_executesql.
If you like, you can try this query at SQL Fiddle. (Note: the SQL Fiddle version replaces the #TQueries table variable with the TQueries table.)
This is what I've managed to piece together cobbled from my original response and improved upon by comments from #AndriyM
DECLARE #sql_prefix NVARCHAR(MAX);
SET #sql_prefix =
N'DECLARE #TResults TABLE (
[Ordinal] INT IDENTITY(1,1),
[ContentItemId] INT
);
DECLARE #max INT, #top INT;
SELECT #max = 10;';
DECLARE #sql_insert_template NVARCHAR(MAX), #sql_body NVARCHAR(MAX);
SET #sql_insert_template =
N'SELECT #top = #max - COUNT(*) FROM #TResults;
INSERT INTO #TResults
SELECT TOP (#top) [Id]
FROM [dbo].[SourceTable]
WHERE
{WherePredicate}
AND NOT EXISTS (
SELECT 1
FROM #TResults AS [tr]
WHERE [tr].[ContentItemId] = [SourceTable].[Id]
)
ORDER BY {OrderByPredicate};';
WITH Query ([Ordinal],[SqlCommand]) AS (
SELECT
[Ordinal],
REPLACE(REPLACE(#sql_insert_template, '{WherePredicate}', [WherePredicate]), '{OrderByPredicate}', [OrderByPredicate])
FROM #TQueries
)
SELECT
#sql_body = #sql_prefix + (
SELECT [SqlCommand]
FROM Query
ORDER BY [Ordinal] ASC
FOR XML PATH(''),TYPE).value('.', 'varchar(max)') + CHAR(13)+CHAR(10)
+N' SELECT * FROM #TResults ORDER BY [Ordinal]';
EXEC(#sql_body);
The basic idea is to use a table variable to hold the results of each query. I create a template for the SQL and replace the values in the template based on what is stored in #TQueries.
Once the entire script is completed I run it with EXEC.
CREATE PROCEDURE [dbo].[sp_SelectRecipientsList4Test] --'6DBF9A01-C88F-414D-8DD9-696749258CEF','Emirates.Description','0','5'
--'6DBF9A01-C88F-414D-8DD9-696749258CEF',
--'121f8b91-a441-4fbf-8a4f-563f53fcc103'
(
#p_CreatedBy UNIQUEIDENTIFIER,
#p_SortExpression NVARCHAR(100),
#p_StartIndex INT,
#p_MaxRows INT
)
AS
SET NOCOUNT ON;
IF LEN(#p_SortExpression) = 0
SET #p_SortExpression = 'Users.Name Asc'
DECLARE #sql NVARCHAR(4000)
SET #sql='
DECLARE #p_CreatedBy UNIQUEIDENTIFIER
SELECT
Name,
POBox,
EmirateName,
TelephoneNo,
RecipientID,
CreatedBy,
CreatedDate,
ID
FROM
(
SELECT Users.Name, Users.POBox, Emirates.Description As EmirateName,
UserDetails.TelephoneNo, AddressBook.RecipientID,AddressBook.CreatedBy, AddressBook.CreatedDate,
AddressBook.ID,
ROW_NUMBER() OVER(ORDER BY '+ #p_SortExpression +') AS Indexing
FROM AddressBook INNER JOIN
Users ON AddressBook.RecipientID = Users.ID INNER JOIN
UserDetails ON Users.ID = UserDetails.UserID INNER JOIN
Emirates ON Users.EmiratesID = Emirates.ID
----WHERE (AddressBook.CreatedBy = #p_CreatedBy)
) AS NewDataTable
WHERE Indexing > '+ CONVERT(NVARCHAR(10), #p_StartIndex) +
' AND Indexing<=(' + CONVERT (NVARCHAR(10),#p_StartIndex ) + ' + '
+ CONVERT(NVARCHAR(10),#p_MaxRows)+') '
EXEC sp_executesql #sql
This query is not giving any error but also not giving any result
please help
Have you tried breaking down the statement, to check if intermediate results are as expected? That's what you do to debug a complex statement...
For example, there's a nested SELECT in there. If you commit that SELECT on its own, does it print the expected values?
Edit: There's a saying about teaching a man to fish. 'ck' and 'n8wrl' have given you fish to eat today, now please practice fishing to feed you tomorrow...
Well, a quick glance of this:
WHERE Indexing > '+ CONVERT(NVARCHAR(10), #p_StartIndex) + ' AND Indexing<=(' + CONVERT (NVARCHAR(10),#p_StartIndex ) +...
looks like you're looking for an impossible condition, not unlike this:
WHERE Indexing > 5 AND Indexing <= 5
So that might be why you're getting no rows, but this proc is ripe for SQL injection attacks too. Building SQL on the fly based on possibly-unvalidated parameters is very dangerous.
You are querying:
'WHERE Indexing > '+ CONVERT(NVARCHAR(10), #p_StartIndex) +
' AND Indexing<=(' + CONVERT (NVARCHAR(10),#p_StartIndex ) + ' + '
and then adding max rows as a string, you can do this much more easily like so:
'WHERE Indexing > '+ CONVERT(NVARCHAR(10), #p_StartIndex) +
' AND Indexing <='+ CONVERT(NVARCHAR(10),#p_StartIndex + #p_MaxRows)
EDIT
The problem with your inner WHERE is that you are passing in the parameter, you need to do
'WHERE (AddressBook.CreatedBy = ''' + CAST(#p_CreatedBy AS CHAR(36)) + ''')'
Are you sure all your joins should be inner joins?
Change sp_executesql to PRINT and see what gets generated (the poor man's debugger)
Besides what all the other people told you,
give me one good reason why you are using sp_executesql over exec? You are not using parameterized statements, you also are not protected from sql injections because you just execute the whole string
This will just bloat the procedure cache everytime this is run and some values change, you will get a new plan every time
Please take a look at Changing exec to sp_executesql doesn't provide any benefit if you are not using parameters correctly and Avoid Conversions In Execution Plans By Using sp_executesql Instead of Exec