Problem: Creating a timer through application will have duplicate the timers when multiple application instances are running.
Problem area: create a timer on cloud.
Can redis be used as timer on cloud? For example, write a record to redis and set ttl(time to live). Once TTL is reached, redis messaging system can be used to receive notification( thereby, executing the task).
The problem is redis seem to be having only publish subscribe mechanism. That means all app instances receive the notification, duplicating the task.
Any suggestions?
Had the same issue a while ago. There are different strategies.
You cloud provider may already have a solution.
Create some sort of Control Database. And lock/check using this db whether the timer is already running for particular instance or tenant.
Send messages to messaging system and make sure duplicates removed.
Run timer process inside lightweight container. Some sort of microservice.
Use any third parties to achieve these.
etc
Every solution has its own pros and cons.
Related
I have a desktop application which should be notified on any table change. So, I found only two solutions which fits well for my case: SqlDependency and SQLCLR. (I would like to know if there is better in .NET stack) I have built the both structure and made them work. I only able to compare the duration of a s̲i̲n̲gl̲e̲ response from SQL Server to the client.
SqlDependency
Duration: from 100ms to 4 secs
SQLCLR
Duration: from 10ms to 150ms
I would like this structure to be able to deal with high rate notifications*, I have read a few SO and blog posts (eg: here) and also am warned from a colleague that on mass requests SqlDependency may go wrong. Here, MS offers something which I didn't get that may be another solution to my problem.
*:Not all the time but for a season; 50-200 requests per sec on 1-2 servers.
On the basis of a high rate of notifications and in parallel with performance, which of these two should I go on with, or is there another option?
Neither SqlDependency (i.e. Query Notifications) nor SQLCLR (i.e. call a Web Service via a Trigger) is going to work for that volume of traffic (50-200 req per sec). And in fact, both options are quite dangerous at those volumes.
The advice given in both linked pages (the one on SoftwareEngineering.StackExchange.com and the TechNet article) are all much better options. The advice on Best way to get push notifications to server from ms sql database (i.e. custom queue table that is polled every few seconds) is very similar to option #1 of the Planning for Notifications TechNet article (which uses Service Broker to handle the processing of the queue).
I like the queuing idea (fully custom or using Service Broker) the best and have used fully custom queues on highly transactional systems (easily the volume you are anticipating) with much success. The pros and cons between these two options (as I see them, of course) are:
Service Broker
Pro: Existing (and proven) framework (can scale and tied into Transactions)
Con: not always easy to configure or administer / debug, can't easily aggregate 200 individual events in 1 second into a single message (will still be 1 message per each Trigger event)
Fully custom queue
Pro: can aggregate many simultaneous trigger events into single "message" to client (i.e. polling service picks up whatever changes happened since last polling), can make use of Change Tracking / Change Data Capture as the source of "what changed" so you might not need to build a queue table.
Con: Is only as scalable as you are able to make it (might be as good, or better, than Service Broker, but highly dependent on your skill and experience to achieve this), needs thorough testing of edge cases to make sure the queue processing doesn't miss, or double-count, events.
You might be able to combine Service Broker with Change Tracking / Change Detection. If there is an easy-enough way to determine the last change processed (change as noted in Change Tracking / Change Data Capture table(s)), then you can set up a SQL Server Agent job to poll every few seconds, and if you find that new changes have come in, then grab all of those changes into a single message to send to Service Broker.
Some documentation to get you started:
Track Data Changes (covers both Change Tracking and Change Data Capture)
SQL Server Service Broker
I cache some data in redis, and reading data from redis if it's exists, otherwise reading data from database and write the data in redis.
I find that there are several ways to update redis after updating database.For example:
set keys in redis to expired
update redis immediately after updating datebase.
put data in MQ and use consumer to update redis.
I'm a little confused and don't know how to choose.
Could you tell me the advantage and disadvantage of each way and it's better to tell me other ways to update redis or recommend some blog about this problem.
Actual data store and cache should be synchronized using the third approach you've already described in your question.
As you add data to your definitive store (i.e. your SQL database), you need to enqueue this data to some service bus or message queue, and let some asynchronous service do the whole synchronization using some kind of background process.
You don't want get into this cases (when not using a service bus and asynchronous service):
Make your requests or processes slower because the user needs to wait until the data is both stored in your database and cache.
Have the risk of a fail during the caching process and not being able to have a retry policy (which is usually a built-in feature in a service bus or some message queues). Also, this failure can end up in a partial or complete cache corruption and you won't be able to automatically and easily schedule some task to fix this situation.
About using Redis key expiration, it's a good idea. Since Redis can expire keys using its built-in mechanism, you shouldn't implement key expiration from the whole background process. If a key exists is because it's still valid.
BTW, you won't be always on this case (if a key isn't expired it means that it shouldn't be overwritten). It might depend on your actual domain.
You can create an api to interact with your redis server, then use SQL CLR to call the call api
I have an app that is running on Google App Engine and uses the Task Queue Api to do some of the heavier lifting in the background. Some of those tasks need to connect to Cloud SQL to do their work. At scale I get too many tasks attempting to connect to Cloud SQL at once. What I need is some sort of data service layer of a shared client so that the tasks themselves aren't making individual connections to Cloud SQL. If anyone has any ideas I'd love to hear them.
Yes, you can actually do this, but it will take a little bit of planning, coding, and configuration in your side.
One idea is to use a Pull Queue (instead of Push Queues). With a pull queue you can schedule your tasks and execute them in a separate module of your application. The hardware of that module can be configured separately from your main module, thus you can avoid too many instances serving requests which in turns will allow you to better use connection pooling.
Of course, depending on the traffic you are getting you might want to decide on how many concurrent backend instances you want running (and connecting to your DB) to avoid/minimize contention.
Easier said than done, but here are two resources that will help you out:
App Engine Modules - https://developers.google.com/appengine/docs/java/modules/
Pull Queues - https://developers.google.com/appengine/docs/python/taskqueue/overview-pull
We have a web app in which a request for a long running or high processor process is called.
We want to create a windows service to off-load this from the IIS servers. We will install this service on multiple machines to lower the wait time for these jobs. One idea we are looking at is serializing the Job object into Sql Server with its JobType as another column.
The job service will claim the job by updating the row with its indicator, this will keep other services from picking it up. Once the job is complete the service removes that entry.
What I am looking for is other, possibly better ideas to accomplish the Job Service Queuing.
I would say this is a great way to handle this issue. The only thing I would add is that while I don't know what the Job object is or how it is created, you might be able to offload this as well. Instead of creating the object and serializing it to the database, simply store the raw data in SQL. Let the Services handle building the Job object themselves from the ground up. That way you cut the serialization out of the mix. However, if this isn't possible, I would say that your solution seems to be the most viable.
If you do go this route, you could look into optimization of your Service offloading. For example, you could wake extra services when the load gets busy and then put some to sleep when the load lightens.
I have been building a multi-user database application (in C#/WPF 4.0) that manages tasks for all employees of a company. I now need to add some functionality such as sending an email reminder to someone when a critical task is due. How should this be done? Obviously I don’t want every instance of the program to be performing this function (Heh each user would get 10+ emails).
Should I add the capability to the application as a "Mode" and then run a copy on the database server in this mode or would it be better to create a new app altogether to perform "Global" type tasks? Is there a better way?
You could create a windows service/wcf service that would poll the database at regular intervals for any pending tasks and send mails accordingly.
Some flag would be needed to indicate whether email is send or not for a particular task.