As an example user performs some update action and three actions need to be performed together:
update database, send email and perform network request to other service. Sending email and performing network request are non-transactional and if one fails and other succeeds we end up in the incorrect state. Are there common patterns for dealing with this?
A common pattern that works for me is to create an Email table as a queue of emails, inserts to this table can be easily included in transactions.
There is another process reading this Email table and sending requests to external processes (smtp server, sendgrid,...) and marking the email as "sent" if we get a success response from external processes. In case there is a failure, the email is not marked as "sent" and could be retried. There could be a situation that an email was sent successfully but there was an error in the network causing a failure response, retrying could send duplicate emails, but that would be rare and should not cause issues.
You can also have a look at database email which is a similar pattern: https://learn.microsoft.com/en-us/sql/relational-databases/database-mail/database-mail?view=sql-server-ver15
We can extend this pattern further with sending network requests with the same idea. You can have a look at this https://microservices.io/patterns/data/transactional-outbox.html
Related
We send monthly emails generated in SQL Server(2017) using sp_send_dbmail to clients containing sensitive data. Recently, there's been a breach where a client received something they shouldn't have received. The cause of this is honestly just bad code, but going forward we want to be able to screen emails more effectively. We are now avoiding the sensitive data issue by asking clients to log in to our system to look at this data instead. Whilst this would be perfectly normal procedure, certain people demand everything in an email. It's a security problem, but the client wants what they want, and the head honcho wants to give the client what they want.
We use Outlook, which has an outbox that nobody really uses anymore as far as I know. Is there a way to get SQL Server to place emails into a shared outbox account so that we can screen emails that contain sensitive data before we send them out?
For initial QA, data for emails containing alerts are generated via stored procedures and sent to QA analysts to check over. This is done by manually setting the recipients to the QA analysts for emails to all clients.
We don't expect them to cover every email as that would be insane, so we ask that they check a random selection to be confident enough that the data is accurate.
Once they are happy, we run the stored procedures again but we pass a parameter so that it cursors through a table that contains every email alert for each client and generates the tailored email for each client and is sent using sp_send_dbmail.
In other words, the email being screened by the QA analysts might not be the same as the email send to clients. This is how the breach occurred.
If we had a way to get these emails into a shared outbox, the QA work can be done on that shared outbox and then we can send the emails. Asking someone to hit send for a big bunch of emails would still be a pain, but perhaps a little less so than having another breach.
AFAIK sp_send_dbmail uses SMTP to send messages. If you want to place messages in the Drafts (or any other) folder of a particular mailbox, you will need to create messages (without sending) using Graph or EWS.
What is Synchronizing client with gmail ? Can anybody give a detailed explanation, because i want to have a better understanding over this concept.
For example, if your client keeps any local cache of the Gmail mailbox data like the Message.Id and labels, or headers, or the entire email. Then in order to update your client you're synchronizing it with Gmail--pulling new updates down to your client. In cases of clients designed for offline use, then synchronizing may also mean pushing local updates back up to the server (e.g. label updates made by client while "offline" that get applied at some later point). That's the general definition of synchronizing.
For the Gmail API specific case, Gmail has a backend mailbox-wide history Id. Any change that affects that account in any way gets a history identifier and most (but not all) history changes affect the state of email messages. Like adding a new message, changing the labels on a message, or deleting a message. Clients of the Gmail API can poll the history Id and find out what's changed since the last time they synchronized and pull down updates to maintain their sync.
I have a Servlet when a request comes it checks for the user id and then if id is not there it creates a new user id in the database. But if I get multiple requests with a very short delay then all those request tend to see that there is not user at the moment and create multiple users with the same name. I just don't want to make the user id field unique to solve this problem. Other than the user id I store some related data as well.
I need to know how to keep a DB locked until one Servlet request is finished processing.
You need to make your servlet code synchronized.
Easy way is to make your servlet implement SingleThreadModel.
http://www.javatpoint.com/SingleThreadModel-interface
But this is not a good approach as your servlet will handle only one thread/request at a time. Good solution is to synchronize the part where you check and generate the uid.
Someone invited me to be a developer on his app. I did not get the email invite?
I have had others invite me and it works (and works well!)
We got around the problem by having an invite sent to another account. That works for now, but I really need the invite to be sent to the correct account.
Gary
The question is borderline off-topic in that it's not usually a programming question, but it comes up so often, from programmers and the customers we support, that I wanted to put an answer here in the hopes that it would be helpful.
I see that it was in your spam folder, which is one of the more common causes of "lost" emails.
In short, when an email is sent from one server, it goes through several other servers and routers on the way to the final user's PC.
Any one of the following conditions could cause a sent email to not reach a recipient:
Blocked Outbound Mail - The sender's company/business/ISP has email scanning software that scans outbound email, and blocks suspicious outbound emails.
Our workplace has this to limit sensitive data being sent out accidentally, to block outbound infected emails sent from infected PCs.
Relay Server Permissions/Configuration - Your code is sending through a relay server that has rules blocking unauthorized use of email relay functionality.
In most shops with good security consciousness, email servers are configured to disallow email relay except for known IP addresses and/or known, explicitly authorized users.
External party blocks the email - The sender's company has been blacklisted.
Blocked Inbound Mail - The receivers company has email scanning software that scans incoming email (similar to #1) to block malicious/non-work related/bad emails from reaching the recipient's inbox.
Receiver's Spam Filtering - The receiver's email inbox has spam mail filtering, which may automatically move the email to a spam folder, delete them, or other action depending on how you have it configured.
Receiver's Inbound Mail Rules - Similar to the above. The recipient may have rules defined that block, delete, or move emails.
Outlook allows this, as do other email clients. Emails can match the patterns set for existing rules and result in false positives that trigger the rule execution.
Human error - the sender sent the email to the wrong email address.
Human error - the recipient accidentally deleted the email and just didn't see it. (You'd be surprised how often I've seen this.)
Bad programming - there was an error sending the email, but the exception handling ignores the error, so nobody ever knows any better.
Only #9 is actually a programming issue, and it's also (in my experience) the least common. Odds are that the problem has nothing to do with code that you'd be writing.
I'm sure there are more, but these are the ones I've seen the most frequently. I'll add as I think of others.
Can someone explain conversation groups in service broker?
Currently, I'm using service broker to send messages from one SQL server to another. On the sending server, I'm trying to correlate the messages so they are processed in serial on the receiving side. Based on the documentation, conversation groups seem to be a perfect fit for this, but on the receiving server, the messages get assigned to a different conversation group from the one I specified when sending the message.
I've search around the web and saw that this behavior seems to be intended (http://social.msdn.microsoft.com/forums/en-US/sqlservicebroker/thread/baf48074-6804-43ab-844a-cb28a6dce02b/), but then I'm confused about the usefulness of the syntax from (http://msdn.microsoft.com/en-us/library/ms178624.aspx)
WAITFOR(
GET CONVERSATION GROUP #conversation_group_id FROM [dbo].[ReceiveQueue]
)
If the conversation group doesn't come across with the message from the sender and messages sent with the same conversation group id don't have the same conversation group id on the receive side, what's the point of the code above?
Conversation groups are a local primitve used for locking. Messages within a conversation group have no order guarantees, and conversation groups do not flow across the wire.
The message order is guaranteed by Service Broker within a conversation. So to preserve the order of corrleated messages in processing, send them on the same conversation.
Conversation groups are needed for groupping a set of conversations that are related to each other. Both GET CONVERSATION GROUP and RECEIVE verbs guarantee that they will lock an entire converstaion group, thus preventing any other thread from processing related messages. For example consider a traveling site. It receives a message with a request to book a holiday package. As a result it initiates a conversation with a hotel booking service and sends a request to reserve a room, it initiates a conversation with an airline reservation service and asks for travel reservation, it initiates a conversation with a car rental agency service and asks for a car reservation. These three new conversation it created are all in the same group with the initial conversation that the request was received on (the application has used the WITH RELATED_CONVERSATION clause of BEGIN DIALOG on all 3 of them). It then commits and proceed to process the messages in the queue. Later responses from these 3 correlated requests start comming in, at pretty much random times. Say the hotel resposnse comes in first. The message gets picked up by the applicaiton and it goes ahead to update the status of the request with the response from the hotel. At the same time, the airline response comes in. If another thread would be allowed to pick it up, it would try to update the status of the same request, thus resulting in blocking or even deadlock against the thread that is processing the hotel response. When the hotel response is processed, the thread commits and thus unlocks the whole conversation group, allowing for any thread (including itself) to pick up the airline response and process it.