I have a scheduled SSIS job which runs daily. The job runs a select query on a SQL 2008 R2 server and then generates a text file. Generally, the job takes less than a minute to complete but sometimes it just takes hours to complete.
Symptoms :
1. The job gets stuck at "Pre-Execute phase is beginning" for just over 10 hours.
2. Then it starts processing the output file.
3. The job then immediately gets stuck at "Rows were provided to a data flow component as input" for another 10 hours.
I don't know what needs to be changed at package level because it works fine 90% of the time. Also, I tried to find a pattern and don't see any. The data being extracted remains almost same. There is no other connection to the table during the job run.
Related
I have an SSIS job which pulls data from one database and pushes into another. Currently the actions are triggered when a record is inserted into a table.
My understanding is using a SQL Server trigger to launch an SSIS Job is not advised. Suggesting to me the preferred route for this use case is to use a recurring schedule.
If I schedule every 10 seconds, will the ETL job launch again if the previous run has not finished? (Is there a better word to describe this behavior in the computing spacing?) If the job relaunches, is there a preferred way to accomplish this behavior?
If I schedule every 10 seconds, will the ETL job launch again if the previous run has not finished?
No. The next run time is computed once the job finishes, based on the "Starting at" and the next interval that meets the cycle interval.
While it is running the "Start Job at Step" option on the SQL Server Management Studio interface will be grayed out.
If you try to kick off the job again forcefully using sp_start_job, you'll get a error message saying it's already running.
I have an SSIS package that is using an ADO NET Source component to select a number of rows from a database and inserting these records into another table.
The default command timeout for this component was 30 seconds, however the query can run for much longer than this 30 second period. Running some tests it seems that this timeout period is only to return the very first row. If it manages to do this then the query can run for 1000s of seconds if needed.
If it fails to return a row in the first 30 seconds then the pre-execute part of the component fails. Which I read is the period between starting to execute the query and the first row being returned.
This seems to conflict with anything I can find on google about this property. They seem to say that the command timeout period is to completely execute the query. That is not what I am seeing.
Can anyone clarify what this command timeout period is actually for?
EDIT:
Links to screenshots of the SSIS component and the property (CommandTimeout) that I am querying about.
Properties
Component
My question isn't about reducing the query time to be within this 30 second timeout period but why executions that take longer than 30 seconds are not timing out.
The 1000 second run of this package was a one of run that moved 17GB~ of data across servers, this isn't a normal run of the package but I was just using it to show that this package is clearly exceeding this 30 second command timeout.
It is a client side setting that the client uses to determine if a query is not responding. The client will then attempt to send a command to the server to cancel the query.
You are correct that the information available is a bit vague, even what is on MSDN. From my personal observations I agree with your findings that if the client starts to receive data from the server before the timeout then it can run for as long as it takes.
I think of it this way. When the first row of data is ready to be returned then the query itself is finished and only the movement of data is left to do. So, by definition the query did not time out.
I have a job that runs every 5 minutes. Every once in a while, if our database load is large, the job will take longer than that. The job will then no longer run, because the next scheduled time is in the past. Is there a way I can get SQL Server to work around this so that it runs again even after one of these long runs?
If you were to have a script that runs continuously, you can spawn a second script from within the script every 5 minutes (without waiting for it to exit). There are many alternative (better) ways to do scheduling in Windows involving custom applications.
But then you will have overlapping script runs if one goes beyond it's 5 minutes, which is probably not what you want. A workaround is to create a temp file when the second script starts and delete it when it's done, and check for its existence in the beginning of the script, if it exists, you exit.
I would like to know what happens if an agent job (with recurring interval) in MS SQL server runs long enough so that it overlaps recurring execution.
According to my tests, paralleled execution does not happen, which is good.
What I'm trying t find out is that will the next execution be ignored because the previous one is not finished yet, OR will it be queued?
By "queued" I mean executing the queued requests immediately after the previous completed discarding the schedule.
Thank you
It will not be queued, it will be skipped. Easy to test: Create a job with a WAITFOR DELAY '00:05:00';, schedule job for < 2 minutes from now, then start the job manually. It will run for 5 minutes, once.
I need your suggestion on tracing the issue.
We are running data load jobs at early morning and loading the data from Excel file into SQL Server 2005 db. When job runs on production server, many times it takes 2 to 3 hours to complete the tasks. We could drill down to one job step which is taking 99% of the total time to finish.
While running the job step (stored procs) on staging environment (with the same production database restored) takes 9 to 10 minutes, the same takes hours on production server when it run at early morning as part of job. The production server always stuck up at the very job step.
I would like to run trace on the very job step (around 10 stored procs run for each user in while loop within the job step) and collect the info to figure out the issue.
What are the ways available in SQL Server 2005 to achieve the same? I want to run the trace only for these SPs and not for certain period time period on production server, as trace give lots of information and it becomes very difficult for me (as not being DBA) to analyze that much of trace information and figure out the issue. So I want to collect info about specific SPs only.
Let me know what you suggest.
Appreciate your time and help.
Thanks.
Use SQL Profiler. It allows you to trace plenty of events, including stored procedures, and even apply filters to the trace.
Create a new trace
Select just stored procedures (RPC:Completed)
Check "TextData" for columns to read
Click the "Column Filters" button
Select "TextData" from the left hand nav
Expand the "Like" tree view, and type in your procedure name
Check "Exclude Rows that Do Not Contain Values"
Click "OK", then "Run"
What else is happening on the server at that time is important when it is faster on other servers but not prod. Maybe you are running into the daily backup or maintenance of statistics or indexes jobs?