I have a job that will create a job for all the databases in the SQL instance. I don't want the jobs to run sequentially. I need multiple databases to run at once, but I also want to make sure that I don't have too many databases running at the same time that might hinder performance on the server.
Is there a way to specify the number of concurrent jobs that can run at the same time or manage the jobs in a way that new jobs won't get started until the number of active jobs is less than what I specify?
You could create a stored procedure that accepts the SQL command(s) and the number [n] of jobs you want to run in parallel and let it create & start n jobs, then go into a loop to poll the msdb tables to see if said jobs are still running and each time it notices there are less than n jobs active (or left) it could start a new job until the entire set of databases has been handled. It should then wait for the last one to finish so the calling process knows everything is processed.
tips:
- Use the #job_id's returned by sp_add_job to check if a job is still running
- Use WITH (NOLOCK) on the system tables to avoid unnecessary locking, it doesn't really matter if you'd use 'dirty reads' when looking for the state of the jobs
- Use WAITFOR DELAY as to only check the tables every 5 seconds or so, otherwise your loop will eat wait too many resources !
Related
I am creating a database medical system and then I came to a point where I am trying to create a notification feature and i will use SQL jobs in it, where the SQL job responsibility is to check some tables and the entities that will find it need to be notified for a change in certain data will put their ids in an entity called Notification and a trigger will be called for the app to check that table and send the notificiation.
what I want to ask is how many SQL jobs can a sql server handle ?
Does the number of running SQL jobs in background affect the performance of my application or the database performance in a way or another ?
NOTE: the SQL job will run every 10 seconds
I couldn't find any useful information online.
thanks in advance.
This question really doesn't have enough background to get a definitive answer. What are the considerations?
Do the queries in your ten-second job actually complete in ten seconds, even when your DBMS is under its peak transactional workload? Obviously, if the job routinely doesn't complete in ten seconds, you'll get jobs piling up.
Do the queries in your job lock up tables and/or indexes so the transactional load can't run efficiently? (You should use SET ISOLATION LEVEL READ UNCOMMITTED; as much as you can so database reads won't lock things unnecessarily.)
Do the queries in your job do a lot of rows' worth of inserts and updates, and so swamp the SQL Server transaction logs?
How big is your server? (CPU cores? RAM? IO capacity?) How big is your database?
If your project succeeds and you get many users, will your answers to the above questions remain the same? (Hint: no.)
You should spend some time on the execution plans for the queries in your job, and try to make them as efficient as possible. Add the necessary indexes. If necessary refactor the queries to make them more efficient. SSMS will show you the execution plans and suggest appropriate indexes.
If your job is doing things like deleting expired rows, you may want to build the expiration in your data model. For example, suppose your job does
DELETE FROM readings WHERE expiration_date >= GETDATE()
and your application does this, relying on your job to avoid getting expired readings.
SELECT something FROM readings
You can refactor your application query to say
SELECT something FROM readings WHERE expiration_date < GETDATE()
and then run your job overnight, at a quiet time, rather than every ten seconds.
A ten-second job is not the greatest idea in the world. If you can rework your application so it will function correctly with a ten-second, ten-minute, or twelve-hour job, you'll have a more resilient production system. At any rate if something goes wrong with the job when your system is very busy you'll have more than ten seconds to fix it.
Is it possible in SQL Server to run several jobs simultaneously under different sessions under same job.
For example, I have N stored procedures to run. They all have to be run under different sessions and start at the same time. I don't want to create N jobs, I want all of them start at the same time under 1 job.
In the past I've had one job create and start several other jobs using the sp_add_job command. If you set the delete level to 3 then the job will then get automatically deleted once it has completed.
The disadvantages are security and monitoring all the jobs.
I don't see any other option than using ssis sql script tasks for different scripts without any link between them and executing them. This will allow to run different SP or sql script to run parallel.Thanks!
I work with an environment that uses Merge Replication to publish a dozen publications to 6 a dozen subscribers every 10 minutes. When certain jobs are running simultaneously, deadlocks and blocking is encountered and the replication process is not efficient.
I want to create a SQL Server Agent Job that runs a group of Merge Replication Jobs in a particular order waiting for one to finish before the next starts.
I created an SSIS package that started the jobs in sequence, but it uses sp_start_job and when run it immediately starts all the jobs so they are running together again.
A side purpose is to be able to disable replication to a particular server instead of individually disabling a dozen jobs or temporarily disabling replication completely to avoid 70+ individual disablings.
Right now, if I disable a Merge Replication job, the SSIS package will still start and run it anyway.
I have now tried creating an SSIS package for each Replication Job and then creating a SQL Server Agent job that calls these packages in sequence. That job takes 8 seconds to finish while the individual packages it is calling (starting a replication job) takes at least a minute to finish. In other words, that doesn't work either.
The SQL Server Agent knows when a Replication job finishes! Why doesn't an SSIS package or job step know? What is the point of having a control flow if it doesn't work?
Inserting waits is useless. the individual jobs can take anywhere from 1 second to an hour depending on what needs replicating.
May be I didn't see real problem but it is naturally that you need synchronization point and there are many ways to create it.
For example you could still run jobs simultaneously but let first job lock a resource that is needed for second, that will wait till resource will be unlocked. Or second job can listen log table in loop (with wait for a "minute" and self cancel after "an hour")...
I am working with SQL Server 2008. Using the Agent, I have created a job and scheduled it to execute every minute.
The job executes a stored procedure that moves data from table XXX, to a temp table, and then eventually into table YYY.
The execution of the job may take more than one minute - since the data is rather large.
Will a second instance of the job be started even though the first instance is still running?
If so, should I mark records in temp table (status = 1) to indicate that those records are being processed by a previous instance of the job?
Is there a way for me to check that an instance of the job is currently running, so that I don't initiate a second instance of the job?
Is there another solution for this that I am unaware of? (throughput is important)
Only one instance of a particular job can run at any one time.
So there is no need to take any particular precautions against another execution of the same job beginning before the first one has stopped.
check this post
How to Prevent Sql Server Jobs to Run simultaneously
How to Prevent Sql Server Jobs to Run simultaneously
As Well HERE
Running Jobs
http://technet.microsoft.com/en-us/library/aa213815(v=sql.80).aspx
If a job has started according to its schedule, you cannot start another instance of that job on the same server until the scheduled job has completed. In multiserver environments, every target server can run one instance of the same job simultaneously.
We have scheduled a number of jobs in SQL Server 2000. We want these jobs to be executed in a sequential order i.e. the failure of one job should prevent the next job from running. Can someone help me on doing this or creating dependency between scheduled jobs.
You could define your jobs as steps of one single job. So you'll can specify on every step if the next step should be executed in case of error.
Rather than combining the jobs in to one single block, it is better to divide in to pieces to simplify the error detection and make the management easier. It gives you to control your process step by step. If your SQL jobs can be executed via batch files, you can use windows task scheduler and define dependencies. But if the subject is more complex ETL process management, it is better to manage this process on a job scheduler.
I've done this in a queue system to cache data where there were 4 or 5 steps involved and had to allow delays for replication between the steps.
It was rather time consuming to implement as there were parent tasks which spawned 1 to n child steps which sometimes needed to be executed in order, sometimes irrelevant.
If you go down this path, you then need to create a location for error messages and process logs.
I highly recommend if in any way it can be created as one job with multiple steps, you should use the existing jobs agent. Each individual step can be configured to exit on fail, continue on fail, email on fail, etc - It's rather flexible.