Have you had experience with SQL queries in Power BI dataflows that run for longer than 10 minutes for one SQL query?
I have migrated an Excel Power Query script to PBI dataflows. One of the migrated queries fails and it fails consistently at 10:00 minutes exactly. But there is not "timeout" error displayed, just a message that says, "Query has cancelled". My query does have the [Command Timeout] property set to 60 minutes, but the property looks like it's being ignored in the PBI service dataflow.
I have written an Excel Power Query script with multiple SQL connections, most connection are stored procedure that return a dataset. One SQL proc takes about 30 minutes to complete. The entire of these queries takes about 50 minutes. We're trying to stage datasets in dataflows and improve performance for reporting for our end users, who are currently pulling datasets from SSRS reports. This dataflow query failure is a major road block.
I'm curious if anyone has had a similar experience. To me it seems ridiculous that a dataflow SQL connection can't run past 10 minutes.
Thanks for your input!
if I run one complicated SQL query it finishes in two minutes. If I run it again it finishes in 30 seconds. So I guess SQL Server has cached something.
Can I check somehow if the query result that I am getting is pulled from some cache or not?
It is a normal condition. SQL Server placed the required data in its cache from the last query execution. So, no more read is needed next time the query is being executed. If there is an error in execution, it shows that on the output window.
I am using a SQL server 2008, which has databases mirrored in Synchronized mode.
I am trying to run some update stored procedures, with some nested joins and it runs fine (Obviously with a reduced performance compared to a server which is not mirrored).
The problem I am facing is that if I select the "show detailed plan" option. The query starts running and it virtually goes to a hung state and doesnt recover. I finally end task the SQL.
I have a public role for the databases and I cant access any stats.
Can you tell me what exactly (or in general) should I ask the DBA to look at?
The details of the SQL server is mentioned below.
Product - SQL Server Enterprise Edtn- 64 bit.
OS - WIndows NT 6.0
Memory -6143 MB
Processor -2
Maximum Server memory - 3072 MB
Minimum server memory - 16 MB
Any help on guiding me to a right direction will be appreciated.
Regards,
Dasso
Because
1) You have activated [Include actual execution plan] option and because
2) There is a WHILE statement
SQL Server will send to client - Sql Server Management Studio the actual execution plan of every SQL statement executed by every iteration of WHILE statement. So, if WHILE includes a simple UPDATE and it executes 100 iterations then Sql Server will send the execution plan of UPDATE 100 times!
You should decrease the number of iterations for WHILE or you could use estimated plans.
I am executing a stored procedure using SQL Server Agent Job in SQL Server 2005.
This job was running fast until yesterday. Since yesterday this job is taking more than 1 hour instead of 2 mins.
I executed the stored procedure in SSMS, it just took less than 1 minute to execute.
I could not figure out why it is taking more than 1 hour when executed as a SQL Server Agent job?
After some time commenting and assuming that the SP performs with the same input parameters and data well when executed in SSMS, I finnaly think I can give a last tip:
Depending on what actions are performed within the SP (e.g. inserting/updating/deleting a lot of data within a loop or cursor), you should set nocount on at the beginning of your code.
set nocount on
If this is not the case or does not help, please add more information, already mentioned in the comments (e.g. all settings of the Job and each Jobstep, what has been logged, what is in the Jobhistory, check SQLerrorlogs, eventlogs,....).
Also take a look at the "SQL Server Logs" maybe you can gather some info here. Also a look into the Application/System eventlo of the Databaseserver is always a good idea.
To get a basic overview you can use the Activitymonitor in SSMS, by selecting the Databaseserver and selecting "Activity monitor" from contextmenu and search for the sql agent.
My last try would be to try to run a sql trace for the agent. In this case you would start a trace and filter e.g. by the user that the SQLAgent Service runs. There are so many options you can set for traces, so I would recommend to google for it, search on MSDN or ask another question here on stackoverflow.
We have a large proc that runs in 88 seconds in SSMS and 30-45 minutes in SQL Server Agent. I added the dbo. prefix on all the table names and now it runs just as fast as SSMS.
I've noticed that SQL Agent jobs ignore the server's MAXDOP setting and run everything with a MAXDOP of 1. If I run a stored procedure in a query windows, it obeys the server settings and uses 4 processes. If I use SQL Agent, any stored procedure I run uses only one process.
I have a similar issue with a script that calls a number of UDFs that I created. The UDF's themselves normally run subsecond under SSMS. Likewise, running the reports I generate with them is bearable under SSMS (30d data in 8s, 365d data in 22s). I've always done NOCOUNT ON with my SQL Agent jobs as they normally generate text files out for pickin up by other processes or Excel and I do not want the extra data at the end, so it was not a solution for me.
In this case, when we run the exact same script under SQL Agent as a job, my times grow exponentially. My 8s script takes 2m30s and my 22s script takes 2h20m. This is the same whether I run it midday with other user activity and jobs or after hours with no user activity, nor jobs or backups running. Our server is idle and at best I get one of the 8 cores being utilized when run. DB is only about 10GB running on SSD with a cached RAID card and 16 of 32GB RAM is free. Since my SQL runs efficiently in SSMS, I am pretty well of the belief that I am hitting a threading limit of some sort. I have researched and tried adjusting MAXDOP just prior to the scripts in the SQL Agent with no luck.
Since this is an activity I want to schedule, it needs to be automated one way or another. I could let these scripts take the hours they need to run as SQL steps in SQL Agent jobs, but I decided to run from command line instead and I get the same performance I see in SSMS.
sqlcmd -S SQLSRVRHost -i "C:\My Script Loc With Spaces.sql" -v MyVar="VarValue" >"C:\MyOutputFile.txt"
So I created a batch script with the SQL jobs run from sqlcmd. Then I run the batch script from a SQL Agent job, so I still have the same management and control in place. My 4 SQL jobs that collectively took over 3 hours to run complete in 1 min and a few seconds from a single batch script executed by SQL Agent.
I hope this helps...
I've got a database View in SQL Server 2008 and when I run it from Query Analyser, it runs within a couple of seconds. When I run it in Excel 2007, it loads the data just fine. But when I run it in Word 2007 on a mail merge, it just seems to take a very long time to run (literally minutes to run).
If anyone has an idea how to solve this, could they let me know?
SQL 2008 will allow you to run a Profiler trace on the query. Start the trace before you kick off the Word query and see how long it takes. My guess is it's the mail-merge that's slow.
If it were slow in Excel too, you'd be looking at your indexes.