I am optimizing my SQL script and when i tried to run it i noticed the following;
Script full query execution (end to end) took more than 2hours to complete
Script query-by-query execution took 44 mins to complete
I'm just wondering, why is my script full query execution took so much runtime than the same script query-by-query execution? Is there some hidden process happening inside the DB? Or any special case for those execution scenario?
Tried to ask our DBA, but he seem to avoid the question maybe he doesn't know. I hope ill got some answers/leads here.
Thanks
Related
I am using SSIS extension 3.16 in VS 2019. I have the following SSIS package
I don't know why when I run it with Debug Mode,. It flagged "Package Execution Completed before all tasks turning green after less than 1 minutes execution.
I have clicked into in those data flows, some indicated data have transferred.
While some was still pending
Queries under those unfinished task is very big and should take very long time to finish (20 mins +).
Could someone share me some light on how to fix it?
Thanks in advance!
I work out the reason was there was a comment on some of those queries. Once done it is all good!
We've got SQL Server 2016 (v13.0.4206.0), by default there is no restrictions for parallelism - any count SQL wants. And it didn't lead any problems... Till now.
For another feature there were written query that unexpectedly raised timeout exception in our application. I was deeply surprised when it was successfully executed with setting up maximum threads per query to 1. Yes, 6 seconds for query is not so good, even accounting to most of time was spent for fetching, but it's far away from 3 minutes timeout!
By the way, executing this query with SQL Server Management Studio works all the time despite of parallelism settings. It seems that something wrong with connection to database, but all other queries works fine, even which much harder then that one.
Our application is built on ASP.NET Core 3.0 (don't know if it matters), database connection is made using System.Data.SqlClient v4.8.0. All I could determine is that there are so much tasks created for this query:
I've tried to watch for execution in sys.dm_os_waiting_tasks (thanks google). I'm not sure I got it right, but it seems that tasks with context_id 0-8 is blocked with those who have context_id 9-16 and vise versa. Obvious example of deadlock, isn't it? But how can SQL Server manage threads to make it without my "help"? Or what am I doing wrong?
Just in case some inappropriate answers:
I won't turn parallelism off (set maximum threads per query to 1) as solution because of some heavy queries in our application;
I don't want to raise Cost Threshold for Parallelism setting because I'm afraid of same problem with another query (guess, a heavier one). So I just want to determine real cause;
Optimizing the query isn't considered (anymore), as according to actual execution plan I can't make it faster - there are enough indexes for it. But I'm ready to rethink after some really weighty arguments.
So, my question is: why does parallelism that I didn't ask for spoil the query execution? And how can I avoid that?
It's true sometimes the engine chooses to use parallel execution (or not to use) which leads to worse performance.
You do not want to control the server option and the cost as you are not sure how this will reflect to other queries, which is understandable.
If you are sure, your query will be execute better without being handle in parallel, you can specify the option just for it using query hints - MAXDOP like this:
SELECT ...
FROM ...
OPTION (MAXDOP 1);
It's easy and you can rollback if needed. Also, you are not affecting other queries.
You are saying that:
Optimizing query isn't considered (anymore), as according to actual execution plan...
The execution plan is sometimes misleading. As a start - you can save your execution plan and open it with SentryOne Plan Explorer - it's free and can give you a better look of what's going on.
Also, if a query is execute for either 3 seconds or 6 minutes, there must be something wrong with it or may be the activity of your database. If it is executed fast in the SSMS always, maybe the engine is using the correct cache plan. I thing it's better to share the query itself and to attach the two plans (serial and parallel) and spend more time tuning it.
I am running my spring application on tomcat7. I use java7, and jtds1.3.1.
I am using sqlserver2014, and the instance is running on a separate machine.
Sometimes, when i run queries through my application, a simple select query (no joins, literally just a select) that takes 1 second if I run it through SQL Server Management Studio, will take 20 minutes or more to complete.
If i check the query on the the sqlserver instance, I see that the total elapsed time keeps incrementing, but the cpu time never increases. Also the status of the query always seems to be 'sleeping'.
I know this is most likely not a problem with SQL Server, as the exact same query finishes instantly when run through SQL Server Management Studio. The fact that the query status seems to constantly be "sleeping" always makes me suspect this.
But beyond this i have no idea how to try and debug what is going on here. Is there anything I can try to change that might help with this issue?
If I run a query that filters on date, if I run the exact same query twice, the first time with my_date_field = '2015-01-01 00:00:00' and the second with my_date_field = '2015-01-01 00:00:00.000', the first one (without microseconds) runs faster. Is that normal? I'm using SQL Server 2008 R2.
Some more background, the execution plans are the same when I run the queries in SSMS. I'm not even sure if I'd get any slowness there. I see the slowness in a PHP application using the native SQL Server driver. I noticed the slowness because there's a loop that runs a query more than once (yeah, I know, to avoid if possible, but hard to avoid here) and the more records you get, the more you notice the slowness. With 100 records, the process runs fast without microseconds and is upwards of 40 seconds with them.
UPDATE
Thanks for the comments that got me digging deeper. Changing the microseconds mad things faster, but it's just a coincidence. After much debugging, I discovered that absolutely any change in the way the query is results in a faster query. As in, I can add a space somewhere or remove a carriage return and it's fast. So my guess is that there's some flawed execution plan that's cached somehow. Only thing that has me a puzzled is that the query parameters are bound, so while I can see how changing the query would fix things, I'm surprised changing the data for that date param fixes it. Maybe I've got that wrong though. Problem is, if it caches the execution plan for the data, how do I get it to update execution plan to something that makes sense?
Turns out all I had to do was run sp_updatestats and performance improved quite a bit.
I am in the process of upgrading our server from one version to other.So for this purpose i need the detail of all the queries executing in single transaction.
I am enabling trace logs for that but that is a tedious process as it contains system related queries also.AWR reports seems to be not working.
I am using oracle 10/11 g.Is there any other way in sql developer to achieve this goal.
Tracing is the only way to see all statements executed. AWR works by sampling active sessions; this significantly cuts down on the overhead and data yet provides plenty of information for performance tuning. But this means that AWR will miss many queries that run very quickly.
To remove recursive queries you can run the trace file through tkprof with the option sys=no.