App A wants to send domain events to App B through a middleware like RabbitMQ.
Let's take the example of one domain event called UserHasBeenRegistered, involving by the creation of the User entity.
A would inform B that this latter should send a welcome email, by sending this event.
I have in mind two workflows:
First:
- App A registers the user and the event is generated.
- App A sends the event directly to B through a queue provided by RabbitMQ
Second:
- App A registers the user and the event is generated.
- App A saves the event in some kind of event store as a database table (if relational) in the same local transaction used for persisting in database this new user.
- An asynchronous scheduler queries the event store, find this new user registration and sends the message through the RabbitMQ's queue.
Do you see the difference?
Yes, one is longer than the other... but the second is far more safe! albeit less performant.
Indeed, what while in the first case, the registration is rollback due to an exception thrown just after the publishing was made? => the mail would be sent whereas the user was not persisted.
This could be fixed by implementing a global XA transaction (two-phases commit), but it is well known that some middleware don't support it.
Therefore, is the second workflow mostly used in critical application?
What are its drawbacks?
I plan to implement one of both solutions for my project.
I had the same task and it was done as a mix of your two workflows:
App A registers the user and the event is generated.
App A sends the event which has ttl set to non-zero value directly to B through a queue provided by RabbitMQ.
App B receive event and send welcome message to user and store flag that welcome message sent.
There are background script which check whether there are newly registered users from last ttl + 1 time interval who doesn't receive messages.
You can remove background script and flag storing and stick with first workflow from you q. The cases when messages lost or any other cases are damn rare (with welcome messages sending it might be 1 failure per 1billion users) and unnecessary application complication may give you more errors.
The second workflow looks also stable, but why you are using RabbitMQ then?
Related
I want to get notified (through email of slack channel) for every Exception occurence in my GAP services.
When I'm trying to create a notification through GCP Error Reporting, it seems like I'm being able to get one notification per incident (and not per occurrence), and that it is also being queried only once a day/hour/month.
I've also tried to create a policy in GCP Logs Monitoring based on log severity, but of course I don't get notified only for exceptions, and the slack notifications just announce that a threshold is being passed, without the actual data I want to get by push.
Any way I can make Error Reporting notify me per each occurance?
Is there any other internal tool by GCP that notify when event occurs? or should I use an external tool such as Epsagon/Operations (formally Stackdriver)?
So it seems like there is no availability for getting a pushed event on each exception occurrence, and since I didn't want to query it by pull (with ereport) or to get only new exceptions (through Stackdriver), I had to go with an external service such as Datadog.
A message is being dropped to Service bus queue with ScheduledEnqueueTimeUTC and Service Bus Connector in Logic app has trigger set to pick messages from queue at 12:05AM EST EveryDay.
Problem: Logic app has picked the same message twice one with Service bus message properties State='Scheduled' and other with state='Active' with same sequenceNumber. May i know when this happens and how can this be solved.
Problem: Logic app has picked the same message twice one with Service
bus message properties State='Scheduled' and other with state='Active'
with same sequenceNumber. May i know when this happens and how can
this be solved.
Here we discovered one of the workarounds that will meet your needs. To pick and send a message only once, we must set our settings to split on as seen below.
NOTE: I tried using Logic app standard, as this option is not available in Consumption plan
Please refer this MS DOC & SO THREAD for more information .
I have multiple microservices that would need to be notified of certain events. All these microservices are running on Cloud Run. Given that Cloud Run can scale down to 0 instances, I’ve assumed that the Push model is better for our situation.
The problem arises with needing to get the notification to multiple endpoints. For example, we have an OrderService, an InventoryService, and PaymentService. The latter two both need to listen for the “OrderPlaced” event. The question is, how can we push this message to both services without having to explicitly call it (e.g. with Task Queues) or creating dependencies within the OrderService? Is there a “Google Cloud” way of solving this issue?
I’ve thought about creating “pull” listeners but the problem is that Cloud Run instances can scale to 0, right, which would effectively not receive events?
When a message is published to a topic, a subscriber is made aware of that message and can be used to push a copy of the message to an application that wishes to process it. If we have multiple applications that wish to be notified when a message is published, we can create multiple subscriptions to the same topic. Each subscription will independently cause a copy of the published message to be pushed to its own registered application in parallel.
I want to send a particular HTTP request (or otherwise communicate a message) to every (dynamic/autoscaled) instance which is currently running for a particular App Engine application.
My goal is to trigger each instance to discard some locally cached data (because I have just modified the underlying data and want them to reload it).
One possible solution is to store a value in Memcache, and have instances check this each time they handle a request to see if they should flush their cache. But this adds latency to every request.
Another possible solution would be to somehow stop all running instances. No fixed overhead, but some impact while instances are restarted.
An even less desirable solution would be to redeploy the application code in order to cause all instances to be stopped. This now adds additional delay on my end as a deployment takes some time.
You could use the management API to list instances for a given version, but I'd suggest that you'd probably want to use something like the PubSub API to create a subscription on each of your App Engine instances. Since each instance has its own subscription, any messages sent to the monitored queue will be received by all instances.
You can create the subscription at startup (the /_ah/start endpoint may be useful), and then delete it at shutdown (using the /_ah/stop endpoint).
I have a C# Web API project on server side and on front-end I have ExtJS 4.2.1 (Javascript framework client).
There is a section in my app where I request to start a long running process (about 5 minutes) and I want to show the user the status of the process being executed.
Basically, the process will run a special calculation for every employee in the database (about 800), so I want to let the user know which Employee is being processed in that moment.
So I was thinking in two ways of doing this, and maybe I don't know if having both is ok.
Use SignalR to show the information of the process in Real Time.
Write to a database table all the process log (every employee that its being processed).
If I use the first approach, if the user close the browser he will loose all the information about the process and if he log into the app again he will only see the actual status.
If I use the second approach, if he log into the app again he could see all the information, and using maybe a timer on client side the data could be refreshed every 5 seconds.
Does anyone have implemented something like this? Any advice is appreciated.
You should use a combination of the two. When you have calculated a employee save the state to the database and publish the change on a service bus.
Let SignalR pick these messages up and forward them to the client. This way the user will see old state when he connects and new state then they arrive with SignalR. I have created a Event aggregator proxy that makes this very easy.
https://github.com/AndersMalmgren/SignalR.EventAggregatorProxy/wiki
Follow the wiki to set it up, here is a demo project
https://github.com/AndersMalmgren/SignalR.EventAggregatorProxy/tree/master/SignalR.EventAggregatorProxy.Demo.MVC4
Live demo
http://malmgrens.org/Signalr/