Currently I'm working on system that gathers data from different websites/apis and stores this data in a SQL Server database. Then reporting service generates reports based on this data.
We have a lot of jobs running in SQL Server agent (each job has some steps, each step can be of type - PowerShell script or SQL...).
Version of SQL Server is 2017.
We have a problem that almost every day there are jobs that starts but never ends (status is "Executing"). This job can not run that long.
Does anybody has an idea how to solve this problem? Or at least how to investigate it?
CPU usage on virtual machine is +- 20%. Memory usage - 50%. So it is not a resource problem.
Related
SQL execution time took more time to complete when database is in the different server compare if the web and database are on the same server
if the database and web server are in one server only: Execution time is 3mins
if the database and web server are in different servers: Execution time is 12mins
What are the things to check?
There are many factors that delay or slowdown the SQL execution, Size of data, network bandwidth, poor execution plan etc are few of them, You can start looking around with these factors and narrow down your analysis. I would suggest to start from comparing execution plan from both servers.
Assuming both the server IIS and DB Server having same specification, same data size then there are likely chances that your DB server configuration is different from what you have on IIS, Check for memory and CPU settings. Do you see same execution plan ?
We recently faced an issue in a server where 12000 concurrent users were trying to access an application but only 120 SQL Server connections were available.
Basic issue I've found is in the architecture of deployment of application and database as below:
DB & App on Same Server
Data and log files of all database whether system or user, are on system drive i.e. C:\
Questions:
By looking what metrics in perfmon or taking what steps can I prove the above points as the basic cause?
Other than the two causes mentioned above, how to correlate metrics/stats in perfmon with a particular SQL Server query?
Currently running SQL Server 2008 R2 SP1 on 64-Bit Windows Server 2008 R2 Enterprise on a Intel dual 8-core processor server with 128 GB RAM and 1TB internal SCSI drive.
Server has been running our Data Warehouse and Analysis Services packages since 2011. This server and SQL instance is not used for OLTP.
Suddenly and without warning, all of the jobs that call SSIS packages that build the data warehouse tables (using Stored Procedures) are failing with "Deadlock on communication buffer" errors. The SP that generates the error within the package is different each time the process is run.
However, the jobs will run fine if SQL Server Profiler is running to trace at the time that the jobs are initiated.
This initially occured on our Development server (same configuration) in June. Contact with Microsoft identified Disk I/O issues, and suggested setting MaxDOP = 8, which has mitigated the deadlock issue, but introduced an issue where the processes can take up to 3 times longer at random intervals.
This just occurred today on our Production server. MaxDOP is currently set to zero. There have been no changes to OS, SQL Server or the SSIS packages in the past month. The jobs ran fine overnight on September 5th, but failed with the errors overnight last night (September 6th) and continue to fail on any retry.
The length of time that any one job will run before failing is not consisent nor is there consistency between jobs. Jobs that take 2 minutes to run to completion previously will fail in seconds, where jobs that normally take 2 hours may run anywhere from 30 - 90 minutes before failing.
Have you considered changing the isoloation level of the database. This can help when parallel reads and writes are happening on the database.
I have created SSIS packages to move data from AS400 to SQL Server which are scheduled daily.some of the packages in sql agent are taking longer duration more than 9 hours to complete.IF I run same package in Business intelligence studio manually, it is completing in less than 4 hours.Due to this problem my schedule packages are not competing on time.please help me to sort out this issue. I am unable to understand why there is a difference in task completion duration between manual interaction and schedule jobs.
My environment is windows server 2003 with sql server 2005 with SP3.please help me to sort out this issue.
The best way to get around this problem is to watch the scheduled task by using some debug statements and messages. For example, have some insert statements in the stored procedures the SSIS package is invoking. This way u will get to know what control is taking more time than expected. First try to isolate the control that is making the difference.
Also, you can invoke the package from command prompt using:-
dtexec /f filename.dtsx
This will print out all the messages in the console at each step as well.
Use SSIS logging in the package to log to a database table. Set logging to record start and end of tasks. By running the package in BIDS and comparing it to the logging when it is run on the server you will see which tasks are taking too long. See http://msdn.microsoft.com/en-us/library/ms138020.aspx for more info on SSIS logging (in sql 2008)
Might it be that the SQL server is less powerful than your client or has more load when you execute the package?
Business intelligence Studio the package is executed on your local client with it's CPU and RAM (I think).
Check what version of DTSEXEC you are using. May be you are using 32-bit version at one place and 64-bit at the other one.
I need to determine if a specific application is running from a SQL Server 2005 job. The issue is that one of our applications we use to send data will hang, causing problems with any subsequent jobs that invokes it. If I can also obtain the CPU time, I can determine if it's likely a hung process.
A list of running applications would be good, but being able to lookup a specific executable name with the CPU time would be fantastic!
Any application launched by a job step will show as being run by the same logon account as the SQL Server Agent. Use a specific service account for the SQL Server Agent that won't be used for any other services. This willallow you to monitor the applications launched from by a job using Task Manager, Performance Monitor, etc.
Try opening the SQL Server Activity Monitor. You can also get some of the information from the stored proc sp_who2.
Have the job run an external script (batch file, KSH script) instead of a TSQL script.
I think the best approach is to run SQL Server Profiler as well as performance monitor and wait for the specified job to run. Then import the perfmon stats into profiler. You can do this from SQL Server profiler by going to File–> Import Performance Data… and point it to your Performance Monitor logs.
You should be able to choose the Process(all) counter to give you a list of all running processes, as well as getting CPU time for the processes. You can then correlate this to the application name and/or hostname from the Profiler logs to see whats going on.
I use the (free) replacement to task manager "Process Explorer" to get a better look at exe's and their dependencies.
Might be worth monitoring your issue with this.
http://technet.microsoft.com/en-us/sysinternals/bb896653.aspx