We are trying to use multi statements in Snowflake Dashboard tiles and do not quite understand the behaviour.
Let's say I create these 2 statements in my tile
SET MyVar = ( SELECT TOP 1 TABLE_NAME FROM DEV_CONTROL.INFORMATION_SCHEMA.TABLES WHERE NOT TABLE_NAME = :Subscription );
SELECT $MyVar;
If I highlight the first line and run it, I get a successful statement that does not return anything.
If I get back to my tile, I see "Statement executed successfully."
If I then go back to my SQL statements and highlights both, then run it, I get the name of the first table.
Going back to my dashboard, I now see the result of the second statement, my table name.
I find this both confusing and incoherent...
The data showed in the tile should reflect ALL the code I entered, not just what I happened to highlight and run the last time I looked at the code?...
Unfortunately, it's not documented well. As you mentioned, the tiles show only the result of the last executed query - at least this is what I observed on my tests.
Using Snowflake Scripting can be helpful here:
DECLARE
MyVar VARCHAR;
Rcount NUMBER;
BEGIN
SELECT TOP 1 TABLE_NAME INTO :MyVar FROM GOKHAN_DB.INFORMATION_SCHEMA.TABLES WHERE NOT TABLE_NAME LIKE 'TEST%' ORDER BY random();
SELECT IFNULL(ROW_COUNT,0) INTO :Rcount FROM GOKHAN_DB.INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = :MyVar;
RETURN :MyVar || ' ' || :Rcount;
END;
The above code will be executed as a block.
Related
I am investigating a problem with the execution speed of an inline table function in SQL Server. Or that's where I thought the problem lay. I came across
T-SQL code is extremely slow when saved as an Inline Table-valued Function
which looked promising, since it described what I was seeing, but I seemed to have the opposite problem - when I passed variables to my function, it took 17 seconds, but when I ran the code of my function in a query window, using DECLARE statements for the variables (which I thought effectively made them literals), it ran in milliseconds. Same code, same parameters - just wrapping them up in an inline table function seemed to drag it way down.
I tried to reduce my query to the minimum possible code that still exhibited the behaviour. I am using numerous existing inline table functions (all of which have worked fine for years), and managed to strip my code down to needing just a call of one existing inline table function to be able to highlight the speed difference. But in doing so I noticed something very odd
SELECT strStudentNumber
FROM dbo.udfNominalSnapshot('2019', 'REG')
takes 17 seconds whereas
DECLARE #strAcademicSessionStart varchar(4) = '2019'
DECLARE #strProgressCode varchar(12)= 'REG'
SELECT strStudentNumber
FROM dbo.udfNominalSnapshot(#strAcademicSessionStart, #strProgressCode)
takes milliseconds! So nothing to do with wrapping the code in an inline table function, but everything to do with how the parameters are passed to a nested function within it. Based on the cited article I'm guessing there are two different execution plans in play, but I have no idea why/how, and more importantly, what I can do to persuade SQL Server to use the efficient one?
P.S. here is the code of the inner UDF call in response to a comment request
ALTER FUNCTION [dbo].[udfNominalSnapshot]
(
#strAcademicSessionStart varchar(4)='%',
#strProgressCode varchar(10)='%'
)
RETURNS TABLE
AS
RETURN
(
SELECT TOP 100 PERCENT S.strStudentNumber, S.strSurname, S.strForenames, S.strTitle, S.strPreviousSurname, S.dtmDoB, S.strGender, S.strMaritalStatus,
S.strResidencyCode, S.strNationalityCode, S.strHESAnumber, S.strSLCnumber, S.strPreviousSchoolName, S.strPreviousSchoolCode,
S.strPreviousSchoolType,
COLLEGE_EMAIL.strEmailAddress AS strEmailAlias,
PERSONAL_EMAIL.strEmailAddress AS strPersonalEmail,
P.[str(Sub)Plan], P.intYearOfCourse, P.strProgressCode,
P.strAcademicSessionStart, strC2Knumber AS C2K_ID, AcadPlan, strC2KmailAlias
,ISNULL([strC2KmailAlias], [strC2Knumber]) + '#c2kni.net' AS strC2KmailAddress
FROM dbo.tblStudents AS S
LEFT JOIN
dbo.udfMostRecentEmail('COLLEGE') AS COLLEGE_EMAIL ON S.strStudentNumber = COLLEGE_EMAIL.strStudentNumber
LEFT JOIN
dbo.udfMostRecentEmail('PERSONAL') AS PERSONAL_EMAIL ON S.strStudentNumber = PERSONAL_EMAIL.strStudentNumber
INNER JOIN
dbo.udfProgressHistory(#strAcademicSessionStart) AS P ON S.strStudentNumber = P.strStudentNumber
WHERE (P.strProgressCode LIKE #strProgressCode OR (SUBSTRING(#strProgressCode, 1, 1) = '^' AND P.strProgressCode NOT LIKE SUBSTRING(#strProgressCode, 2, LEN(#strProgressCode)))) AND
(P.strStudentNumber NOT IN
(SELECT strStudentNumber
FROM dbo.tblPilgrims
WHERE (strAcademicSessionStart = #strAcademicSessionStart) AND (strScheme = 'BEI')))
ORDER BY P.[str(Sub)Plan], P.intYearOfCourse, S.strSurname
)
Expanding on #Ross Pressers comment, this might not really be an answer, but demonstrates what is happening (a bit), with my understanding (which could be wrong!) of what is happening...
Run the setup code at the end and then....
Execute the following with query plan on (Ctrl-M)... (note: depending on the random number generator you may or may not get any results, that does not affect the plan)
declare #one varchar(100) = '379', #two varchar(200) = '726'
select * from wibble(#one, #two) -- 1
select * from wibble('379', '726') -- 2
select * from wibble(#one, #two) OPTION (RECOMPILE) -- 3
select * from wibble(#one, #two) -- 4
Caveat. The following is what happens on MY system, your mileage may vary...
-- 1 (and -- 4) are the most expensive.
SQL Server creates a generic plan as it does not know what the parameters are (yes they are defined, but the plan is for wibble(#one, #two) where, at that point, the parameter values are "unknown")
https://www.brentozar.com/pastetheplan/?id=rJtIRwx_r
-- 2 has a different plan
Here, sql server knows what the parameters are, so can create a specific plan, which is quite different to --1
https://www.brentozar.com/pastetheplan/?id=rJa9APldS
-- 3 has the same plan as --2
Testing this further, adding OPTION (RECOMPILE) gets SQL Server to create a specific plan for the specific execution of wibble(#one, #two) so we get the same plan as --2
--4 is there for completeness to show that after all that mucking about the generic plan is still in place
So, in this simple example we have a parameterised TVF being called with identical values, that are passed either as parameters or inline, producing different execution plans and different execution times as per the OP
Set up
use tempdb
GO
drop table if EXISTS Orders
GO
create table Orders (
OrderID int primary key,
UserName varchar(50),
PhoneNumber1 varchar(50),
)
-- generate 300000 with randon "phone" numbers
;WITH TallyTable AS (
SELECT TOP 300000 ROW_NUMBER() OVER(ORDER BY (SELECT NULL)) AS [N]
FROM dbo.syscolumns tb1,dbo.syscolumns tb2
)
insert into Orders
select n, 'user' + cast(n as varchar(10)), cast(CRYPT_GEN_RANDOM(3) as int)
FROM TallyTable;
GO
drop function if exists wibble
GO
create or alter function wibble (
#one varchar(4) = '%'
, #two varchar(4) = '%'
)
returns table
as
return select * from Orders
where PhoneNumber1 like '%' + #one + '%'
and PhoneNumber1 like '%' + #two + '%'
or (SUBSTRING(#one, 1, 1) = '^' AND PhoneNumber1 NOT LIKE SUBSTRING(#two, 2, LEN(#two)))
and (select 1) = 1
GO
Problem was overcome (I wouldn't say "fixed") by following up on Ross Presser's observation about the complexity of udfProgressHistory. This sucked data from a table tblProgressHistory which was joined to itself. The table is added to annually. I think this year's additional 2K records must have caused the sudden cost-hike when using a particular execution plan. I deleted >2K redundant records and we're back to sub-second execution.
I'm creating a stock market database and am stumped that the following works correctly EXCEPT for the last select that returns results (after which the select does not change on subsequent loops). I've tried to simplify the code as follows, thanks in advance for feedback (I'm still noob):
Three tables:
BuyOrders
SellOrders
MatchedOrders
Stored procedure to process a NewBuyOrder:
Insert NewBuyOrder to BuyOrders;
While (NewBuyOrder.SharesRemaining > 0 )
SELECT TOP 1
FROM SellOrders
WHERE SellOrders.Price <= NewBuyOrder.Price
ORDER BY SellOrders.Price, SellOrders.TimePlaced;
IF NewBuyOrder.SharesRemaining < SellOrders.SharesAvailable
UPDATE SellOrders.SharesAvailable = [difference];
UPDATE BuyOrders = 0;
INSERT INTO MatchedOrders;
SET NewBuyOrder.SharesRemaining = 0;
BREAK;
ELSE
UPDATE SellOrders = 0;
UPDATE BuyOrders = [difference];
INSERT INTO MatchedOrders;
SET NewBuyOrder.SharesRemaining = [difference];
CONTINUE;
In hope it might help someone else, I found the issue . . . I'm using local variables to store the matched SellOrderID. As such if the Select returns no match on a second pass through then the local variables were not getting updated (and hence erroneously reused in subsequent while loops until the If kicked in).
So I put a SET SellOrders.ID = 0 into the WHILE loop before the Select then below the Select added a IF SellOrders.ID = 0 and inside that a SET NewBuyOrder.SharesRemaining = 0 and BREAK (then made the first IF above into an ELSE IF).
I need to revisit the process to see if I can make it more elegant but would sincerely welcome thoughts on better ways to accomplish a process for matching the best available counteroffers in sequence. I've read but don't know much about cursors, plus think it transactionally superior not to SELECT a prioritized table of all matches rather than using my iterative loop -- but also have read suggestions not to use loops in SQL. Comments?
In addition I note the following: By itself a Select with no results returns a null set. Thus my original plan was to Select into my SP local variables and then use an IF EXISTS. I assume the local variable exists upon instantiation (even with no value) but am surprised that after a Select into the local variable with no results also did not fail an IF NULL test (i.e. presumably NULL cannot be inserted into a variable). What then is the value of an instantiated local variable with no value -- Blank?
I have this Oracle code that I need to convert in SQL Server but need help understanding what exactly it is doing . Since I have always avoided cursors it is still a mystery to me how they are used . please see the code below . This is placed in an insert trigger
CURSOR c1(table_name1 IN VARCHAR2)
IS
SELECT
a.begin, a.end, a.isnotactive, a.isactive, MIN(g.age) minage
FROM alltables a
LEFT OUTER JOIN people g ON (g.ageid = a.ageid)
WHERE table_name = table_name1
c1x_rec c1%ROWTYPE;
c1_rec c1%ROWTYPE;
I am particularly unsure about the following 3 lines . What exactly is it doing ? Where does the table_name1 gets its value from ?
WHERE table_name = table_name1
c1x_rec c1%ROWTYPE;
c1y_rec c1%ROWTYPE;
OPEN c1(table_name1) - this will open the cursor
FETCH c1 INTO variable - this is will fetch data into variable
You need to create variable, that must match with your select statement in your cursor, to fetch the data. For this, you use %ROWTYPE - that attribute provides a record type that represents a row. For example, your variables c1x_rec, c1y_rec have begin, end, isnoactive, isactive, minage fields, with each field declared as equivalent column type in alltables or people table.
I need some help from a MS SQL Master...
Short version:
When I execute a Conditional Where followed by a Contains, my query delays 1 minute (In its normal execution, it takes 200 milliseconds).
With this query, everything works fine:
Where
Contains(table.product_name, #search_word)
But with a Conditional Where, it takes 1 minute to execute:
Where
(#ExecuteWhereStatement = 0 Or (Contains(table.product_name, #search_word))
Long Version:
I'm using a stored procedure that receives some parameters. This Stored Procedure query a really large table, but everything is indexed properly and the query goes very well so far.
The main query is a little big, so I want to make the WHERE clause more smart possible, to avoid repeat multiple times the same statement.
The whole idea of the DataBase, is a history of purchases made by the State. So this query involves 3 tables:
Table 1 (table_purchase) - The purchase itself
id_purchase int (PK)
date_purchase datetime
buyer_code int (Nullable)
Table 2 (table_purchase_product) - The Items of a Purchase
id_product int (PK)
id_purchase int (FK of table_purchase)
product_quantity int (Nullable)
product_name varchar(255) (Nullable) (Full-Text-Indexed)
product_description varchar(2000) (Nullable) (Full-Text-Indexed)
id_product_bid_winner int (FK of table_product_bid)
Table 3 (table_product_bids) - The Bids for Each product of a Purchase
id_product_bid int (PK)
id_product int (FK of table_purchase_product)
product_brand varchar(255) (Nullable) (Full-Text-Indexed)
bid_value decimal (20,6)
So basicly, We have a "Purchase", that has several "Products (or Items)", and each "Product" has some "Bids (or Prices)"
And there is the Bad Girl (The SQL Stored Procedure):
ALTER PROCEDURE [dbo].[procPesquisaFullText]
#search_date datetime,
#search_word varchar(8000),
#search_brand varchar(255),
#only_one_bid bit = 0,
#search_buyer_code int = 0,
#quantityFrom decimal(20,6) = 0,
#quantityTo decimal(20,6) = 0
AS
BEGIN
SET NOCOUNT ON;
Declare #ExecuteWordSearch AS bit;
if (#uasg != 0 And #search_word = '')
begin
Set #ExecuteWordSearch = 0;
Set #search_word = 'nothing';
end
else
begin
Set #ExecuteWordSearch = 1;
end
Declare #ExecuteBrandSearch AS bit;
if (#search_brand = '')
begin
Set #ExecuteBrandSearch = 0;
Set #search_brand = 'nothing';
end
else
begin
Set #ExecuteMarcaSearch = 1;
end
begin
SELECT
pp.id_product,
pp.id_purchase,
pp.description
FROM
table_purchase_product pp
inner join table_purchase p on p.id_purchase = pp.id_purchase
WHERE
(p.date_purchase >= #search_date)
and (#search_buyer_code = 0 or (l.buyer_code = #search_buyer_code))
and (#quantityFrom = 0 or (li.product_quantity >= #QuantityFrom))
and (#quantityTo = 0 or (li.product_quantity <= #QuantityTo))
and (contains(pp.product_description, #search_word) or contains(pp.product_name, #search_word))
and (#only_one_bid = 0
or ((Select COUNT(*) From table_product_bid Where table_product_bid.id_product = pp.id_product) = 1))
and (#ExecuteBrandSearch = 0 Or (exists(
select 1
from table_product_bid ppb
where ppb.id_product_bid = pp.id_product_bid_winner
and contains(ppb.product_brand, #search_brand)
)
))
ORDER BY p.date_purchase DESC
end
END
So far, so good...
In the beginning I set two variables, used inside the query.
The first, verify if the user specified a "Buyer Code" AND didn't specify a "Search Word" (So, not the Product's description nor the Product's name is verified)
The second, verify if the user specified a "Specific Brand". If so, then the Winning Bid's BRAND is verified to match the users one.
Observation: You'll notice that when the "Search Words" is empty, I set them to "nothing". I do it because if the search term in the Contains is empty, it throws me a exception, even when it's not executed (I tested it in another query, absolutely isolated too)
As You can see, my user is able to search for:
- "Products" of Some Distinct Buyer "Purchase" (passing the #search_buyer_code parameter)
- A "Product" that contains a distinct word in its name or description
- A "Product" that has the Winner Bid of a specific Brand
- A "Product" that has only 1 bid at all
- A "Product" with a maximum and minimum quantity
And You'll notice that I used a lot of Conditions INSIDE the Where, producing a very dynamic Where, instead of using a "BIG If Else" statement, and repeating a lot of code. (I guess some "Googlers" will land here looking for Conditionally Wheres, and If so, I'm glad to help!)
Ok, so everything works veeery great at all. The query executes flawless. But here is the strange, damn, tricky issue:
If I want the user to be able to specify only a "Buyer Code" for Purchase, but No Word to Search of the Product using the code above (which is the first piece of code in the stored procedure does):
Changing from:
and (contains(pp.product_description, #search_word) or contains(pp.product_name, #search_word))
To:
and (#ExecuteWordSearch = 0 Or (contains(pp.product_description, #search_word) or contains(pp.product_name, #search_word)))
The query delays near 1 minute! (the execution is about 200 milliseconds for the query above).
But WHY??? I Use the same Logic of in all "Conditionally Wheres". I also use the same logic of having a flag/variable to indicate when execute the Where clause in the Word Search and the Brand Search, but the Brand Search works PERFECTLY! So Why, WHY only when I use the condition followed by a Contains my query delays 1 minute????
And this issue is not related with the amount of data, because I tried removing the entire Contains condition, allowing a lot of data to return, and it takes 1 second maximum...
Ow, It's a Microsoft SQL Server 2008 R2.
Thanks already for You read so far!
I cannot find the documentation I had around a very similar issue, but it sounded so familiar, I at least wanted to share what I remembered. Part of the issue is that for Sql Server, the full-text search engine is separate from the regular query execution engine, and so when you mix the two, in some cases, performance can tank. This is particularly true when the condition is an 'OR' rather than and 'AND'. (I remember hitting this exact situation). Conditional ANDs worked fine. But for OR, it's as if each condition gets evaluated repeatedly row by row.
Among the workarounds, one is, as already suggested, create your sql dynamically before execution.
Another would be to break the full-text and non-full text conditions into two search functions (literally UDF's) and then do whatever is needed (INTERSECT, EXCEPT, etc) with the two resultsets.
Try changing your WHERE clause to use a CASE statement, e.g.:
WHERE
CASE
WHEN #ExecuteWhereStatement = 0 THEN 1
WHEN #ExecuteWhereStatement = 1 THEN
CASE
WHEN CONTAINS([table].product_name, #search_word) THEN 1
ELSE 0
END
END = 1;
I am using SDAC components to query a SQL Server 2008 database. It has a recordcountproperty as all datasets do and it also has the FetchAll property (which I think it is called packedrecords on clientdatasets). Said that, I got a few questions:
1 - If I set FetchAll = True the recordcount property returns ok. But in this case, when I have a large database and my query returns a lot of lines, sometimes the memory grows a lot (because it is fetching all data to get the recordcount of course).
2 - If I set FetchAll = False, the recordcount returns -1 and the memory does not grow. But I really need the recordcount. And I also wanna create a generic function for this, so I dont have to change all my existent queries.
What can I do to have the recordcount working and the memory usage of the application low in this case?
Please, do not post that I dont need recordcount (or that I should use EOF and BOF) because I really do and this is not the question.
I thought about using a query to determine the recordcount, but it has some problems since my query is going to be executed twice (1 for recordcount, 1 for data)
EDIT
#Johan pointed out a good solution, and it seems to work. Can anybody confirm this? I am using 1 TMSCconnection for every TMSQuery (because i am using threads), so I dont think this will be a problem, will it?
MSQuery1.FetchAll := False;
MSQuery1.FetchRows := 10;
MSQuery1.SQL.Text := 'select * from cidade';
MSQuery1.Open;
ShowMessage(IntToStr(MSQuery1.RecordCount)); //returns 10
MSQuery1.Close;
MSQuery2.SQL.Text := 'SELECT ##rowcount AS num_of_rows';
MSQuery2.Open;
ShowMessage(MSQuery2.FieldByName('num_of_rows').AsString); //returns 289
EDIT 2*
MSQuery1 must be closed, or MSQuery2 will not return the num_of_rows. Why is that?
MSQuery1.FetchAll := False;
MSQuery1.FetchRows := 10;
MSQuery1.SQL.Text := 'select * from cidade';
MSQuery1.Open;
ShowMessage(IntToStr(MSQuery1.RecordCount)); //returns 10
//MSQuery1.Close; <<commented
MSQuery2.SQL.Text := 'SELECT ##rowcount AS num_of_rows';
MSQuery2.Open;
ShowMessage(MSQuery2.FieldByName('num_of_rows').AsString); //returns 0
Run your query as normal, than close the query
MSQuery1.SQL.Text := 'select * from cidade';
MSQuery1.Open;
MSQuery1.Close;
You need the close otherwise SQL-server has not closed the cursor yet, and will not register the query as 'completed'.
and run the following query right afterwards:
SELECT ##rowcount AS num_of_rows
This will select the total number of rows your last select read.
It will also select the number of rows your update/delete/insert statement affected.
See: http://technet.microsoft.com/en-us/library/ms187316.aspx
Note that this variable is per connection, so queries in other connections do not affect you.
I use ODAC and I believe SDAC inherits from the same base classes and works the same way as ODAC. In ODAC, there is an option called QueryRecCount under Options in your query component. Look for TCustomDADataSet.Options.QueryRecCount in your help file.
Setting QueryRecCount = True and FetchAll = False will reduce your memory usage and give you the record count. But SDAC will run a second query in the background to get the record count so it does add a little bit of extra time to your query.
Take a look at the Devart forum entry at http://www.devart.com/forums/viewtopic.php?t=8143.