I am creating a system where users can setup mailings to go out at specific times. Before I being I wanted to get some advice. First, is there already a .Net component that will handle scheduling jobs (either running another application or calling a URL) that will do what I am suggesting (Open Source would be cool)? If there isn’t, is it better to schedule a job in SQL and run some sort of script, create a .Net service that will look at an xml file or db for schedules, or have an application create scheduled tasks? There could be a ton of tasks, so I am thinking creating scheduled tasks or SQL jobs might not be a good idea.
Here may be a typical scenario; a user wants to send a newsletter to their clients. The user creates the newsletter on a Saturday, but doesn’t want it to go out until Monday. The user wants that same e-mail to go out every Monday for a month.
Thanks for looking!
Check out Quartz.NET
Quartz.NET is a full-featured, open
source job scheduling system that can
be used from smallest apps to large
scale enterprise systems.
If you want to use the readily available services in Windows itself, check out this article A New Task Scheduler Task Library on CodeProject on how to create scheuled tasks in Windows from your C# application.
You probably have more flexibility and power if you use C# and scheduled tasks in Windows, rather than limiting yourself to what can be done in SQL Server. SQL Server Agent Jobs are great - for database specific stuff, mostly - maintenance plans and so forth.
You can build your own windows service that schedules and executes jobs. Be sure to make good abstractions. In a similar project, I have used an abstraction where scheduling items are abstracted as Jobs composed of tasks. For example, sending newsletter may be a job whereas sending newsletter to each subscriber can be considered as a task. Then you need to run the job and tasks in defined threading models preferably using Threadpool threads or Task Parallel Library. Be sure to use asynchronous API for IO whenever possible. Also separate your scheduling logic from the abstractions. so that the scheduling logic can execute arbitrary types of jobs and its inclusive tasks.
Related
Let's there is a master package and several tasks run in it on a daily basis, I want to specifically determine when those tasks have finished like table load completed and cuble load completed, these steps run daily but I have to show this in a SQL table that this particular day table load started at this time and ended at this like etc
SSIS event handlers are the simplest means of turning an SSIS script
into a reliable system that is auditable, reacts appropriately to
error conditions, reports progress and allows instrumentation and
monitoring your SSIS packages. They are easy to implement, and provide
a great deal of flexibility. Rob Sheldon once again provides the easy,
clear introduction.
You can use on PostExecute and when the tasks runs successfully :
Or you can use containers and then use precedence constraints with success and failure
I have a situation where I have a job that runs every day (Job A), job that runs every 2 days (Job B) and another job that runs every weekend (Job C). I need to make sure that Job A runs before Job B. If Job A does not run appropriately then i don't want Job B to run. The same thing applies to Job C. Anyone have any thoughts on how to go about this?
Appreciate any help
I have used a product called SQL Sentry to do what you are trying to do. SQL Sentry has a lot of other advanced monitoring and control functionality (like killing jobs that hang, queuing low priority jobs, etc). Here is their website https://sentryone.com/platform/sql-server-performance-monitoring.
This is a quote from one of their advertising:
19. Chaining and Queuing
Did you ever wish you could find just a few more hours in your
maintenance window, or need to have jobs run in a particular sequence?
The advanced chaining features in SQL Sentry Event Manager can assure
that interdependent jobs run in the proper order without wasting time
or resources.
Chaining
SQL Sentry Event Manager allows you to chain
SQL Agent Jobs, Windows Tasks, or Oracle Jobs across servers. You can
enforce dependencies and automate workflow throughout your entire
enterprise, even across platforms! The graphical chaining interface
allows you to design the workflow using variables such as completion,
success, or failure. More details are available in the User Guide, but
take a few minutes to watch our two video tutorials on chaining -
Graphical Chaining Interface and Advanced Chaining.
I need to make sure that Job A runs before Job B. If Job A does not run appropriately then i don't want Job B to run. The same thing applies to Job C.
Create all jobs A,B,C and schedule only job A..At the end of job A ,success event ,Call job B like below
EXEC dbo.sp_start_job N'Weekly Sales Data Backup' ;
GO
Now the same things applies to job c,call job c on success event of job B..
I would go with this approach..You also can go with an approach of insert success ,failure values into a table and ensure job b or c reads those values before starting
I have a database hosted on Azure. I have an MVC 4 website were users log in and interact with the database. I need something that I can use to traverse through a table in my database, check for certain conditions and then make the required changed to my database. What framework or coding language could I use to achieve this? My hope is that I could have this script run continually or at certain time intervals.
There are multiple ways of achieving this and they all depend on your need for scale/resources/etc of this script.
You can code the job in whatever language (including batch if you utilize sqlcmd or some such)
You can schedule the job to run anywhere you want, including a machine in your office, data center, etc.
You can utilize newly released Aditi's job scheduler (it's free, called Scheduler, and is available in Azure app store)
I am using quartz.net for job scheduling. I have a table in my database JobQueue. Quartz.net reads from this JobQueue and executes the jobs. Everything works fine. Now there is a new requirement that i need to extend the job scheduling for more than one client. This means Quartz.net has to be configured to read from more than one database. How to handle this scenario? I have one service written using Quartz.net which should schedule jobs for multiple clients by reading from their respective databases. Any help will be greatly appreciated. Thanks.
How sure how it will scale up, but it is possible to create multiple Appdomains and have a different scheduler hosted inside each appdomain.
If you are reading the quartz settings from the app.config file, then you would probably need to change some settings such as the connectionstring at runtime.
If I want to conduct some database operations on a scheduled basis, I could:
Use SQL Server Agent (if SQL Server) to periodically call the stored procedure and/or execute the T-SQL
Run some external process (scheduled by the operating system's task scheduler for example) which executes the database operation
etc.
Two questions:
What are some other means of accomplishing this
What decision criteria should one use to decide the best approach?
Thank you.
Another possibility is to have a queue of tasks somewhere, and when applications that otherwise use the database perform some operation, they also do some tasks out of the queue. Wikipedia does something like this with its job queue. The scheduling isn't as certain as with the other methods, but you can e.g. put off doing housekeeping work when your server happens to be heavily loaded.
Edit:
It's not necessarily better or worse than the other techniques. It's suitable for tasks that do not have to be performed by any specific deadline, but should be done "every now and then", or "soon, but not necessarily right now".
Advantages
You don't need to write a separate application or set up SQL Server Agent.
You can use any criteria you can program to decide whether to run a task or not: immediately, once a certain time has passed, or only if the server is not under heavy load.
If the scheduled tasks are ones like optimising indices, then you can do them less frequently when they are less necessary (e.g. when updates are rare), and more frequently when updates are common.
Disadvantages
You might need to modify multiple applications to cooperate correctly.
You need to ensure that the queue doesn't build up too much.
You can't reliably ensure that a task runs before a certain time.
You might have long periods where you get no requests (e.g. at night) where deferred/scheduled tasks could get done, but don't. You could combine it with one of the other ideas, having a special program that just does the jobs in the queue, but you could just not bother with the queue at all.
You can't really rely on external processes. All 'OS' based solutions I've seen failed to deliver in the real world: a database is way more than just the data, primarily because of the backup/restore strategy, the high availability strategy, the disaster recoverability strategy and all the other 'ities' you pay for in your SQL Server license. An OS scheduler based will be an external component completely unaware and unintegrated with any of them. Ie. you cannot back/restore your schedule with your data, it will not fail over with your database, you cannot ship it to a remote disaster recovery site through your SQL data shipping channel.
If you have Agent (ie. not Express edition) then use Agent. Has a long history of use and the know how around it is significant. The only problem with Agent is its dependence on msdb that makes it disconnect from the application database and thus does not play well with mirroring based availability and recoverability solutions.
For Express editions (ie. no Agent) the best option is to roll your own scheduler based on conversation timers (at least in SQL 2k5 and forward). You use conversations to chedule yourself messages at the desired moment and rely on activated procedures to run the tasks. They are transactional and integrated with your database, so you can rely on them being there after a restore and after a mirroring or clustering fail over. Unfortunately the know how around how to use them is fairly skim, I have several articles about the subject on my site rusanu.com. I've seen systems replicating a fair amount of Agent API on Express relying entirely on conversation timers.
I generally go with the operating systems scheduling method (task scheduler for Windows, cron for Unix).
I deal with multiple database platforms (SQL Server, Oracle, Informix) and want to keep the task scheduling as generic as possible.
Also, in our production environment we have to get a DBA involved for any troubleshooting / restarting of jobs that are running in the database. We have better access to the application servers with the scheduled tasks on them.
I think the best approach for the decision criteria is what the job is. If it's a completely internal SQL Server task or set of tasks that does not relate to the outside world, I would say a SQL Job is the best bet. If on the other hand, you are retrieving data and then doing something with it that is inherently outside SQL Server, very difficult to do in T-SQL or time consuming, perhaps the external service is the best bet.
I'd go with SQL Server Agent. It's well integrated with SQL Server; various SQL Server features use Agent (Log Shipping, for instance). You can create an Agent job to run one or more SSIS packages, for instance.
It's also integrated with operator notification, and can be scripted, or else executed through SMO.