I am executing an asynchronous query against Snowflake and getting back a query_id. I want to query Snowflake to determine the execution status of the query, so I will know when it is safe to retrieve the results. What is the best way to do this? I am ruling out the below methods, as they each have limitations.
QUERY_HISTORY Function - I only want the status for 1 query. This function returns multiple queries, which is inefficient. Additionally, it has a row limit, so if my query was more than 10,000 queries ago, it will not be returned. Also, the bigger value I set for result_limit, the longer the function takes to run. A call with the value of 100 took 5 seconds to complete, while a value of 10,000 took multiple minutes, before I gave up and aborted the query.
QUERY_HISTORY View - The documentation states that this view has up to a 45 minute latency in data. I need to know the status in real time.
You should just be able to run
select *
from table(information_schema.query_history())
where query_id = ?
Related
I heard select queries that use a limit turn nondeterministic. I tried a simple query several times using limit and offset, and got the same results each time. Why is that?
If a user repeats a query that has already been run, and the data in the table(s) hasn’t changed since the last time that the query was run, then the result of the query is the same. However, this is not guaranteed though the factors listed in the below link are met and hence non-deterministic.
https://docs.snowflake.com/en/user-guide/querying-persisted-results.html#retrieval-optimization
In our snowflake, we have multiple users accessing the database. And sometimes when query takes long time people are cancelling the query from UI.
I was wondering if there is a way to get information about who has cancelled the query.
Like we have table QUERY_HISTORY to get information about the query.
Do we have a similar table to get information of query cancelled?
Only the user who executed the query can cancel it via the Abort button in the UI. Other ways to cancel a query would be to issue one of the following functions:
SYSTEM$CANCEL_QUERY
SYSTEM$CANCEL_ALL_QUERIES
SYSTEM$ABORT_SESSION
SYSTEM$ABORT_TRANSACTION
ALTER WAREHOUSE ... ABORT ALL QUERIES
ALTER USER ... ABORT ALL QUERIES
Any one of the above functions that are issued would be a query in the QUERY_HISTORY as well.
A cancelled query will have an error of 604, and I expect that the error code is different if the query has timed out (which by default is set to 2 days).
The question is whether there is a table that stores cancelled queries like QUERY_HISTORY, I think. The answer is that even a cancelled query will be in QUERY_HISTORY.
I have a coding problem I can't solve.
The requirement is show progress of a sql query. I am using Spring Boot and Angularjs, so the idea is to show progress of sql to UI, not need to be real time, but preferred. Basically user click on button on UI, it will trigger API to retrieve data from db and then return the completed data to UI.
We have a computation extensive sql which will takes long time to finish. When the row we want to retrieve is about 10 million, it roughly takes about 15 minutes. We want to show the progress of the SQL so User have a idea of how long it will take. So the idea is to check how many rows has completed:
Say 1 million retrieved, then it should return 10% and so on. Next 1 million, then 20%.
I have no idea how to approach this. Need some suggestion.
Thanks in Advance.
Assuming this is one long select call, you can run the following query to get the stats on a long-running statement:
select * from v$session_longops where time_remaining > 0 and sid = ???
sid value should be replaced with the oracle session id of the session that is running the long query. You can determine that from looking in v$session. You will need to execute the above query in a separate thread (or whatever a concurrent unit of execution is in spring).
Thanks Everyone. I've tried to use OldProgrammer's solution, but failed to do so. Somehow I just can't retrieve Hibernate sesison ID, and all related post I found were either too old or just manually create new session which I don't want.
I did a work around to this problem, not a solution though.
So what I did is that for that complex SQL, first I did total count, then chunk it into 100 chunks. Then with each chunk finish(Pagination), I will update the completion percentage. Meanwhile, on the UI, I will have time-interval to check the progress of this SQL from a different API endpoint, currently I set it to every 2 seconds.
So the outcome is like this, for fairly small amount of rows, it will jump from 2% to 6% for every API call; for large amount, only after few API progress check, it will go from 1% to 2%.
This is not a clean solution, but it did its purpose, it will give User some idea of how long it will take to finish.
Goal --> Try to find the execution time the query will take to display result before executing the query.
Detail Description -->
I am trying to run an simple query , for Example one shown below,
I am trying to find out how much actual time the query will take before running the query
SELECT top 1 *
FROM table_name
The answer isn't fixed. You can run the query, have it take 5 seconds, repeat it and have it take less than one because some part of the query or results were cached.
Contention on tables makes a difference too. Selecting has to wait if it uses an index that's being updated.
All you can do is experiment to make estimates, potentially also using control L to get an estimated execution plan.
I have an NHibernate Query (which is populating an EXTJS grid)
It's firing 2 queries off to the database, one to get the record count (for paging purposes) and the other to get the top N rows to fill the grid with.
From code, I'm consistently getting an exception on the Select count(*) statement.
NHibernate.Exceptions.GenericADOException:
Failed to execute multi criteria[SQL:
SELECT count(*) as y0_ FROM RecordView this_ inner join ProcessesView
process1_ on this_.ProcessId=process1_.Id inner join BusinessModelsView
businessmo3_ on process1_.BusinessModelId=businessmo3_.Id inner join BatchesView
batch2_ on this_.BatchId=batch2_.Id WHERE this_.ProcessId = ?;
] ---> System.Data.SqlClient.SqlException: Timeout expired.
The timeout period elapsed prior to completion of the operation or the server
is not responding.
However if I take that exact query and drop it into an SSMS window, and run it, it executes in a <1 second.
Is NHibernate doing anything "funny" under the hood here. Are there execution plan/cache issues. I'm at a complete loss as to why this is occurring.
Whenever I encountered this error, the reason was locking (never performance). There was two sessions opened (accidently). Both started transaction and one of them locked the table.
The problem could be some not disposed session, or "unintended" singleton... holding opened session.
This answer is not as straigth forward as I wish, but I am sure about the direction. Because I experienced the same (and was guilty)
BTW: as Oskar Berggren found out from you, 30 secods timeout would be related to the <property name="command_timeout">30</property>. I am sure, if you will provide 60, 120 ... it will be not enough because of lock
Your two queries are not handled in the same way by SQL SERVER
your NH query has been compiled on its first execution, based on table statistics and on the first value of the parameter. The generated query plan will then be used for all subsequent calls, witout considering the parameter value
your SQL query (where, I guess, you replace the ? with an actual value) gets a different compilation for each value, based on statistics, and on the value.
Your first NH compilation might have produced a query plan, effective for the first value, but not in the general case.
First, I would suggest that :
you count on a projection (say on the main table id), as it is slightly more effective than count(*), allowing the DB to work only on indexes when possible
you check that you don't miss any index necessary to your query
you check that all your table statistics are up to date
If this does not improve execution time, this post offers some options (recompile might be the good one) :
Query executed from Nhibernate is slow, but from ADO.NET is fast