I heard select queries that use a limit turn nondeterministic. I tried a simple query several times using limit and offset, and got the same results each time. Why is that?
If a user repeats a query that has already been run, and the data in the table(s) hasn’t changed since the last time that the query was run, then the result of the query is the same. However, this is not guaranteed though the factors listed in the below link are met and hence non-deterministic.
https://docs.snowflake.com/en/user-guide/querying-persisted-results.html#retrieval-optimization
Related
Just sharing my experience, and maybe can i get some clarifications/confirmations.
I'am executing a certain script which will return Customer Information.
This script has at least 5 to 10 joined tables(with pivots-joined tables), and most likely to return at least 100k records.
The thing is, when i tried to add filter on my WHERE clause - "STATUS != 'CLOSED'", my query runs forever.
When i change it to "STATUS != 'CLOSE'", my query runs at least 2 minutes.
Also, please be informed that "CLOSED" is not existing.
Is this some kind of bug? Or am i missing something?
most likely because it needs to scan all records and output them,where in CLOSE example it probably excludes a lot of records.
Without sharing a query, execution plans it is hard to comment here.
Also a side note having 10 table joins, pivots and other stuff lumped in one query is usually not recommended.
Goal --> Try to find the execution time the query will take to display result before executing the query.
Detail Description -->
I am trying to run an simple query , for Example one shown below,
I am trying to find out how much actual time the query will take before running the query
SELECT top 1 *
FROM table_name
The answer isn't fixed. You can run the query, have it take 5 seconds, repeat it and have it take less than one because some part of the query or results were cached.
Contention on tables makes a difference too. Selecting has to wait if it uses an index that's being updated.
All you can do is experiment to make estimates, potentially also using control L to get an estimated execution plan.
I have one simple query which has multiple columns (more than 1000).
When i run with single column it gives me result in 2 seconds with proper index seek, logical read, cpu and every thing is under thresholds.
But when i select more than 1000 columns it takes 11 mins for the result and gives me key lookup.
You folks have you faced this type of issue?
Any suggestion on that issue?
Normally, I would suggest to add those columns in the INCLUDE fields of your non-clustered index. Adding them in the INCLUDE removes the LOOKUP in the execution plan. But as everything with SQL Server, it depends. Depending on how the table is used i.e, if you're updating the table more than just plain SELECTing on it, then the LOOKUP might be ok.
If this query is run once per year, the overhead of additional index is probably not worth it. If you need quick response time, that single time of the year when it needs to be run, look into 'pre executing' it and just present the result to the user.
The difference in your query plan might be because of join elimination (if your query contains JOINs with multiple tables) or just that the additional columns you are requesting do not exist in your currently existing indexes...
I have an NHibernate Query (which is populating an EXTJS grid)
It's firing 2 queries off to the database, one to get the record count (for paging purposes) and the other to get the top N rows to fill the grid with.
From code, I'm consistently getting an exception on the Select count(*) statement.
NHibernate.Exceptions.GenericADOException:
Failed to execute multi criteria[SQL:
SELECT count(*) as y0_ FROM RecordView this_ inner join ProcessesView
process1_ on this_.ProcessId=process1_.Id inner join BusinessModelsView
businessmo3_ on process1_.BusinessModelId=businessmo3_.Id inner join BatchesView
batch2_ on this_.BatchId=batch2_.Id WHERE this_.ProcessId = ?;
] ---> System.Data.SqlClient.SqlException: Timeout expired.
The timeout period elapsed prior to completion of the operation or the server
is not responding.
However if I take that exact query and drop it into an SSMS window, and run it, it executes in a <1 second.
Is NHibernate doing anything "funny" under the hood here. Are there execution plan/cache issues. I'm at a complete loss as to why this is occurring.
Whenever I encountered this error, the reason was locking (never performance). There was two sessions opened (accidently). Both started transaction and one of them locked the table.
The problem could be some not disposed session, or "unintended" singleton... holding opened session.
This answer is not as straigth forward as I wish, but I am sure about the direction. Because I experienced the same (and was guilty)
BTW: as Oskar Berggren found out from you, 30 secods timeout would be related to the <property name="command_timeout">30</property>. I am sure, if you will provide 60, 120 ... it will be not enough because of lock
Your two queries are not handled in the same way by SQL SERVER
your NH query has been compiled on its first execution, based on table statistics and on the first value of the parameter. The generated query plan will then be used for all subsequent calls, witout considering the parameter value
your SQL query (where, I guess, you replace the ? with an actual value) gets a different compilation for each value, based on statistics, and on the value.
Your first NH compilation might have produced a query plan, effective for the first value, but not in the general case.
First, I would suggest that :
you count on a projection (say on the main table id), as it is slightly more effective than count(*), allowing the DB to work only on indexes when possible
you check that you don't miss any index necessary to your query
you check that all your table statistics are up to date
If this does not improve execution time, this post offers some options (recompile might be the good one) :
Query executed from Nhibernate is slow, but from ADO.NET is fast
I have a query with about 6-7 joined tables and a FREETEXT() predicate on 6 columns of the base table in the where.
Now, this query worked fine (in under 2 seconds) for the last year and practically remained unchanged (i tried old versions and the problem persists)
So today, all of a sudden, the same query takes around 1-1.5 minutes.
After checking the Execution Plan in SQL Server 2005, rebuilding the FULLTEXT Index of that table, reorganising the FULLTEXT index, creating the index from scratch, restarting the SQL Server Service, restarting the whole server I don't know what else to try.
I temporarily switched the query to use LIKE instead until i figure this out (which takes about 6 seconds now).
When I look at the query in the query performance analyser, when I compare the ´FREETEXT´query with the ´LIKE´ query, the former has 350 times as many reads (4921261 vs. 13943) and 20 times (38937 vs. 1938) the CPU usage of the latter.
So it really is the ´FREETEXT´predicate that causes it to be so slow.
Has anyone got any ideas on what the reason might be? Or further tests I could do?
[Edit]
Well, I just ran the query again to get the execution plan and now it takes 2-5 seconds again, without any changes made to it, though the problem still existed yesterday. And it wasn't due to any external factors, as I'd stopped all applications accessing the database when I first tested the issue last thursday, so it wasn't due to any other loads.
Well, I'll still include the execution plan, though it might not help a lot now that everything is working again... And beware, it's a huge query to a legacy database that I can't change (i.e. normalize data or get rid of some unneccessary intermediate tables)
Query plan
ok here's the full query
I might have to explain what exactly it does. basically it gets search results for job ads, where there's two types of ads, premium ones and normal ones. the results are paginated to 25 results per page, 10 premium ones up top and 15 normal ones after that, if there are enough.
so there's the two inner queries that select as many premium/normal ones as needed (e.g. on page 10 it fetches the top 100 premium ones and top 150 normal ones), then those two queries are interleaved with a row_number() command and some math. then the combination is ordered by rownumber and the query is returned. well it's used at another place to just get the 25 ads needed for the current page.
Oh and this whole query is constructed in a HUGE legacy Coldfusion file and as it's been working fine, I haven't dared thouching/changing large portions so far... never touch a running system and so on ;) Just small stuff like changing bits of the central where clause.
The file also generates other queries which do basically the same, but without the premium/non premium distinction and a lot of other variations of this query, so I'm never quite sure how a change to one of them might change the others...
Ok as the problem hasn't surfaced again, I gave Martin the bounty as he's been the most helpful so far and I didn't want the bounty to expire needlessly. Thanks to everyone else for their efforts, I'll try your suggestions if it happens again :)
This issue might arise due to a poor cardinality estimate of the number of results that will be returned by the full text query leading to a poor strategy for the JOIN operations.
How do you find performance if you break it into 2 steps?
One new step that populates a temporary table or table variable with the results of the Full Text query and the second one changing your existing query to refer to the temp table instead.
(NB: You might want to try this JOIN with and without OPTION(RECOMPILE) whilst looking at query plans for (A) a free text search term that returns many results (B) One that returns only a handful of results.)
Edit It's difficult to clarify exactly in the absence of the offending query but what I mean is instead of doing
SELECT <col-list>
FROM --Some 6 table Join
WHERE FREETEXT(...);
How does this perform?
DECLARE #Table TABLE
(
<pk-col-list>
)
INSERT INTO #Table
SELECT PK
FROM YourTable
WHERE FREETEXT(...)
SELECT <col-list>
FROM --Some 6 table Join including onto #Table
OPTION(RECOMPILE)
Usually when we have this issue, it is because of table fragmentation and stale statistics on the indexes in question.
Next time, try to EXEC sp_updatestats after a rebuild/reindex.
See Using Statistics to Improve Query Performance for more info.