I've got a question which occurs when I was using the WITH-clause in one of my script. The question is easy to pointed out I wanna use the CTE alias multiple times instead of only in outer query and there is crux.
For instance:
-- Define the CTE expression
WITH cte_test (domain1, domain2, [...])
AS
-- CTE query
(
SELECT domain1, domain2, [...]
FROM table
)
-- Outer query
SELECT * FROM cte_test
-- Now I wanna use the CTE expression another time
INSERT INTO sometable ([...]) SELECT [...] FROM cte_test
The last row will lead to the following error because it's outside the outer query:
Msg 208, Level 16, State 1, Line 12 Invalid object name 'cte_test'.
Is there a way to use the CTE multiple times resp. make it persistent? My current solution is to create a temp table where I store the result of the CTE and use this temp table for any further statements.
-- CTE
[...]
-- Create a temp table after the CTE block
DECLARE #tmp TABLE (domain1 DATATYPE, domain2 DATATYPE, [...])
INSERT INTO #tmp (domain1, domain2, [...]) SELECT domain1, domain2, [...] FROM cte_test
-- Any further DML statements
SELECT * FROM #tmp
INSERT INTO sometable ([...]) SELECT [...] FROM #tmp
[...]
Frankly, I don't like this solution. Does anyone else have a best practice for this problem?
Thanks in advance!
A CommonTableExpression doesn't persist data in any way. It's basically just a way of creating a sub-query in advance of the main query itself.
This makes it much more like an in-line view than a normal sub-query would be. Because you can reference it repeatedly in one query, rather than having to type it again and again.
But it is still just treated as a view, expanded into the queries that reference it, macro like. No persisting of data at all.
This, unfortunately for you, means that you must do the persistance yourself.
If you want the CTE's logic to be persisted, you don't want an in-line view, you just want a view.
If you want the CTE's result set to be persisted, you need a temp table type of solution, such as the one you do not like.
A CTE is only in scope for the SQL statement it belongs to. If you need to reuse its data in a subsequent statement, you need a temporary table or table variable to store the data in. In your example, unless you're implementing a recursive CTE I don't see that the CTE is needed at all - you can store its contents straight in a temporary table/table variable and reuse it as much as you want.
Also note that your DELETE statement would attempt to delete from the underlying table, unlike if you'd placed the results into a temporary table/table variable.
Related
I have an INSERT trigger of a table, where one field of the table contains a comma-separated list of key-value pairs, that are separated by a :
I can select this field with the two values into a temp table easily with this statement:
-- SAMPLE DATA FOR PRESENTATION ONLY
DECLARE #messageIds VARCHAR(2000) = '29708332:55197,29708329:54683,29708331:54589,29708330:54586,29708327:54543,29708328:54539,29708333:54538,29708334:62162,29708335:56798';
SELECT
SUBSTRING(value, 1,CHARINDEX(':', value) - 1)AS MessageId,
SUBSTRING(value, CHARINDEX(':', value) + 1, LEN(value)-SUBSTRING(value,0,CHARINDEX(value,':'))) AS DeviceId
INTO #temp_messages
FROM STRING_SPLIT(#messageIds, ',')
SELECT * FROM #temp_messages
DROP TABLE #temp_messages
The result will look like this
29708332 55197
29708329 54683
29708331 54589
29708330 54586
29708327 54543
29708328 54539
29708333 54538
29708334 62162
29708335 56798
From here I can join the temp table to other tables and insert some of the results into a third table.
Inside the trigger I can get the messageIds with a simple SELECT statement like
DECLARE #messageIds VARCHAR(2000) = (SELECT ProcessMessageIds FROM INSERTED)
Now I create the temp table (like described above) and process my
INSERT INto <new_table> SELECT col1, col1, .. FROM #temp_messages
JOIN <another_table> ON ...
Unfortunately this will only work for single row inserts. As soon as there is more than one row, my SELECT ProcessMessageIds FROM INSERTED will fail, as there are multiple rows in the INSERTED table.
I can process the rows in a CURSOR but as far as I know CURSORS are a no-go in triggers and I should avoid them whenever it is possible.
Therefore my question is, if there is another way to do this without using a CURSOR inside the trigger?
Before we get into the details of the solution, let me point out that you would have no such issues if you normalized your database, as #Larnu pointed out in the comment section of your question.
Your
DECLARE #messageIds VARCHAR(2000) = (SELECT ProcessMessageIds FROM INSERTED)
statement assumes that there will be a single value to be assigned to #messageIDs and, as you have pointed out, this is not necessarily true.
Solution 1: Join with INSERTED rather than load it into a variable
INSERT INTO t1
SELECT ...
FROM t2
JOIN T3
ON ...
JOIN INSERTED
ON ...
and then you can reach INSERTED.ProcessMessageIds without issues. This will no longer assume that a single value was used.
Solution 2: cursors
You can use a CURSOR, as you have already pointed out, but it's not a very good idea to use cursors inside a trigger, see https://social.msdn.microsoft.com/Forums/en-US/87fd1205-4e27-413d-b040-047078b07756/cursor-usages-in-trigger-in-sql-server?forum=aspsqlserver
Solution 3: insert a single line at a time
While this would not require a change in your trigger, it would require a change in how you insert and it would increase the number of db requests necessary, so I would advise you not to choose this approach.
Solution 4: normalize
See https://www.simplilearn.com/tutorials/sql-tutorial/what-is-normalization-in-sql
If you had a proper table rather than a table of composite values, you would have no such issues and you would have a much easier time to process the message ids in general.
Summary
It would be wise to normalize your tables and perform the refactoring that would be needed afterwards. It's a great effort now, but you will enjoy its fruits. If that's not an option, you can "act as if it was normalized" and choose Solution 1.
As pointed out in the answers, joining with the INSERTED table solved my problem.
SELECT INTAB.Id,
SUBSTRING(value, 1,CHARINDEX(':', value) - 1)AS MessageId,
SUBSTRING(value, CHARINDEX(':', value) + 1, LEN(value)-SUBSTRING(value,0,CHARINDEX(value,':'))) AS DeviceId
FROM INSERTED AS INTAB
CROSS APPLY STRING_SPLIT(ProcessMessageids,',')
I never used "CROSS APPLY" before, thank you.
Given an arbitrary select query, how can I save its results into a temporary table?
To simplify things let's assume the select query does not contain an order by clause at the top level; it's not dynamic SQL; it really is a select (not a stored procedure call), and it's a single query (not something that returns multiple result sets). All of the columns have an explicit name. How can I run it and save the results to a temp table? Either by processing the SQL on the client side, or by something clever in T-SQL.
I am not asking about any particular query -- obviously, given some particular SQL I could rewrite it by hand to save into a temp table -- but about a rule that will work in general and can be programmed.
One possible "answer" that does not work in general
For simple queries you can do
select * into #tmp from (undl) x where undl is the underlying SQL query. But this fails if undl is a more complex query; for example if it uses common table expressions using with.
For similar reasons with x as (undl) select * into #tmp from x does not work in general; with clauses cannot be nested.
My current approach, but not easy to program
The best I've found is to find the top level select of the query and munge it to add into #tmp just before the from keyword. But finding which select to munge is not easy; it requires parsing the whole query in the general case.
Possible solution with user-defined function
One approach may be to create a user-defined function wrapping the query, then select * into #tmp from dbo.my_function() and drop the function afterwards. Is there something better?
More detail on why the simple approach fails when the underlying uses CTEs. Suppose I try the rule select * into #tmp from (undl) x where undl is the underlying SQL. Now let undl be with mycte as (select 5 as mycol) select mycol from mycte. Once the rule is applied, the final query is select * into #tmp from (with mycte as (select 5 as mycol) select mycol from mycte) x which is not valid SQL, at least not on my version (MSSQL 2016). with clauses cannot be nested.
To be clear, CTEs must be defined at the top level before the select. They cannot be nested and cannot appear in subqueries. I fully understand that and it's why I am asking this question. An attempt to wrap the SQL that ends up trying to nest the CTEs will not work. I am looking for an approach that will work.
"Put an into right before the select". This will certainly work but requires parsing the SQL in the general case. It's not always obvious (to a computer program) which select needs to change. I did try the rule of adding it to the last select in the query, but this also fails. For example if the underlying query is
with mycte as (select 5 as mycol) select mycol from mycte except select 6
then the into #x needs to be added to the second select, not to the one that appears after except. Getting this right in the general case involves parsing the SQL into a syntax tree.
In the end creating a user-defined function appears to be the only general answer. If undl is the underlying select query, then you can say
create function dbo.myfunc() returns table as return (undl)
go
select * into #tmp from dbo.myfunc()
go
drop function dbo.myfunc
go
The pseudo-SQL go indicates starting a new batch. The create function must be executed in one batch before the select, otherwise you get a syntax error. (Just separating them with ; is not enough.)
This approach works even when undl contains subqueries or common table expressions using with. However, it does not work when the query uses temporary tables.
1 As I known, CTE is more used for Query readability, not for performance.
Without talking about indexing, should we just use the temp table in all cases instead of CTE because there is no performance benefit anyway and temp table can achieve the same outcome as CTE?
2 But I still cannot get this: Temp Tables are physically created in the Tempdb database while CTE is not materialized. They are created in memory and got disposed of straight away after the next statement.
Should we have fast better performance to access the same data in memory instead of materialized disk space?
Temp Tables are physically created in the Tempdb database while CTE is
not materialized. They are created in memory and got disposed of
straight away after the next statement.
This is wrong. CTE is not created, it's not a structure containing data. It's just like a view with the only difference that its definition is not saved as an object. As a consequence, it cannot be reused in other statements. It's just another form of writing a derived table.
For example, when you write
with USAcusts as
(
select custid, companyname
from sales.customers
where country = N'USA'
)
select * from USAcusts;
It's the same as the code below:
select *
from
(
select custid, companyname
from sales.customers
where country = N'USA'
) t
None of CTE or derived table is "created in memory", exactly like in view case, SQL Server expands the definition of the table expression and queries the underlying objects directly.
If you are looking for performance, always use temp table.
I will only use CTE for these reasons:
very small and 1 time use temp data
recursive query
Why is this legal:
DECLARE #Party TABLE
(
PartyID nvarchar(10)
)
INSERT INTO #Party
SELECT Name FROM
(INSERT INTO SomeOtherTable
OUTPUT inserted.Name
VALUES ('hello')) [H]
SELECT * FROM #Party
But the next block gives me an error:
WITH Hey (Name)
AS (
SELECT Name FROM
(INSERT INTO SomeOtherTable
OUTPUT inserted.Name
VALUES ('hello')) [H]
)
SELECT * FROM Hey
The second block gives me the error "A nested INSERT, UPDATE, DELETE, or MERGE statement is not allowed in a SELECT statement that is not the immediate source of rows for an INSERT statement.
It seems to be saying thst nested INSERT statements are allowed, but in my CTE case I did not nest inside another INSERT. I'm surprised at this restriction. Any work-arounds in my CTE case?
As for why this is illegal, allowing these SELECT operations with side effects would cause all sorts of problems I imagine.
CTEs do not get materialised in advance into their own temporary table so what should the following return?
;WITH Hey (Name)
AS
(
...
)
SELECT name
FROM Hey
JOIN some_other_table ON Name = name
If the Query Optimiser decided to use a nested loops plan and Hey as the driving table then presumably one insert would occur. However if it used some_other_table as the driving table then the CTE would get evaluated as many times as there were rows in that other table and so multiple inserts would occur. Except if the Query Optimiser decided to add a spool to the plan and then it would only get evaluated once.
Presumably avoiding this sort of mess is the motivation for this restriction (as with the restrictions on side effects in functions)
I have a SQL SELECT statement that will create its results in XML Format. I would like to INSERT into another table (Log via insert) every time a row is selected from the original SELECT Statement. Is there anyway to include an INSERT Statement inside of a SELECT Statement?
SELECT cs_ads_StoreLocations.LocationGUID, *Bunch of Selects AS Distance
FROM *Bunch of Inter Joins*
WHERE *Bunch of conditions*
ORDER BY *You don't want to know*
FOR XML AUTO
INSERT INTO cs_ads_StoreLog (LocationGUID) VALUES (*DISTINCT cs_ads_StoreLocations.LocationGUID from select statement above*)
This is just some sample code that has the INSERT outside of the SELECT statement. I need something that has it inside the SELECT Statement or another way of running an INSERT
Just to clarify. This sample code is part of a Stored Proc
INSERT will actually accept a SELECT as its input. You can just use this command (your SELECT copied as a subquery, though you could probably refactor it to return those distinct GUIDs without a subquery...but I'd need to see the whole monster).
insert into (cs_ads_StoreLog)
select distinct
a.LocationGUID
from
(SELECT
cs_ads_StoreLocations.LocationGUID, *Bunch of Selects AS Distance
FROM
*Bunch of Inter Joins*
WHERE
*Bunch of conditions*
ORDER BY
*You don't want to know*
) a
Then, you can just return the SELECT that you have (it'll be cached, so you don't need to worry about performance hits too much).
Perhaps dynamic SQL would solve this? Example here.
Or maybe you could store the values in a table var, and do your insert on that.