The put(...) method of GAE's memcahe API accepts as an argument (in one of it's overloaded implementations) a SetPolicy argument. In the Javadocs here it sais that if you chose "ADD_ONLY_IF_NOT_PRESENT" as policy it's, and I quote:
"useful to avoid race conditions."
My questions are:
what happends with an expired value that was set with the same key? If I add to memcache something like (key=1, value=whatever), then this entry expires, and then I try to add (key=1, value=whatever2) using ADD_ONLY_IF_NOT_PRESENT is whatever2 added to cache or not?
What does it mean "useful for race conditions"? More specifically, does it mean that if I use put(...) with ADD_ONLY_IF_NOT_PRESENT SetPolicy I am no longer required to use getIdentifiable and putIfUntouched in order to avoid race conditions when adding stuff concurrentlly to the memcache?
If the value expires, it's not in memcache anymore, so the RPC will set it.
If you do a get, then do a put only if nothing was there, you've introduced a race condition whereby someone else might've put the data while you were checking. Doing a single operation avoids this.
Related
I think that have seen in many occasions that a DynamoDB conditional put throws ConditionalCheckFailedException but succeeds. Usually in this scenario, the request takes quite long (~10s) to finish, but I can see that the request is updated despite the fact that a ConditionalCheckFailedException is thrown (and the it took few seconds).
By the way I don't force any timeout on the DDB request.
Is this a bug, or some DDB conditional put contract that I misunderstand? Has anyone experienced this issue?
Answering this late to inform others:
ConditionCheckFailedException but item is persisted:
This typically happens when you save an item to DynamoDB, DynamoDB acknowledges the write request but the response gets lost on the return path which can happen for multiple reasons, keeping in mind that DynamoDB is one of the largest distributed systems in the cloud.
This causes the SDK timeout to exceed while awaiting a response, which then triggers an SDK retry. When the write request is retried, the condition now evaluates to False as the item already exists, which in turn throws a ConditionCheckFailedException, which can cause confusion.
When I receive a ConditionCheckFailedException I typically do a strongly consistent GetItem request for the item to ensure it exists with the values I expect and move on.
Per angular_devise docs, https://github.com/cloudspace/angular_devise, using Auth._currentUser is frowned upon.
Auth._currentUser
Auth._currentUser will be either null or the currentUser's object
representation. It is not recommended to directly access
Auth._currentUser, but instead use Auth.currentUser().
Why is this a bad idea?
I'm currently using it in an application because it allows me to be more concise and add functionality with fewer lines of code.
Underscored properties and methods are considered private, this means that this part of API can be changed without notice and their usage can lead to unforeseen circumstances.
Since getting current user is asynchronous operation, promise-based Auth.currentUser() should be used instead.
The last but not the least concern is that calls to Auth.currentUser can be spied in specs, this improves testability.
I would like to make a db.put() operation in my Google App Engine service as resilient as possible, trying to maximize the likelihood of success even in the event of infrastructure issues or overload. What I have come up with at the moment is to catch every possible exception that could occur and to create a task that retries the commit if the first attempt fails:
try:
db.put(new_user_record)
except DeadlineExceededError:
deferred.defer(db.put,new_user_record)
except:
deferred.defer(db.put,new_user_record)
Does this code trap all possible error paths? Or are there other ways db.put() can fail that would not by caught by this code?
Edit on March 28, 2013 - To clarify when failure is expected
It seems that the answers so far assume that if db.put() fails then it is because the datastore is down. In my experience of having run fairly high-workload applications this is not necessarily a requirement. Sometimes you run into workload-specific API bottlenecks, sometimes the slowness of one API causes the request deadline to expire in another. Even though such events have a low frequency, their number can be sizable if traffic is high. These are the situations I am trying to cover.
I wouldn't say this is the best approach - whatever caused the original exception is just likely to happen again. What I would do for extra resilience is first load the record to be saved into memcache and in the event of an exception with the put (any exception) it could attempt a certain number of retries (for example 3) with a short sleep between each attempt. Depending on your application this could be either a synchronous operation or using deferred tasks it could be done asynchronously using the data in memcache.
Finally I'd actually do a query on the record in the data store even if there wasn't an exception to confirm the row has actually been written.
Well, i don't think that it is a good idea to try such a fallback at all. If the datastore is down, its down and youre out of luck (shouldn't happen frequently :)
Some thoughts to your code:
There are way more exceptions that could be raised during a put-opertation (like InternalError, Timeout, CommittedButStillApplying, TransactionFailedError)
Some of them don't mean that the put has failed. (ie. CommittedButStillApplying just means the put-operation is delayed). With your approach, you would end up having that entry twice in the datastore after your deferred call succeeds.
Tasks are limited to ~100KB (total size, not payload). If your payload is close to or above that limit, the deferred-api will automatically try to
serialize your payload to the datastore in order to keep the task itself below that limit. If the datastore is really unavailable, this will fail, too.
So its probably better to catch datastore errors, and inform your user that his request failed.
Its all good to retry, however use exponential backoff and most important proper transaction use so that fail xoesnt end up o a partial write.
I'm developing a Silverlight app that makes multiple async requests to a number of web services. I want a modal "loading" dialog to stay active until all the requests have completed. I'm managing the situation by using a counter variable that gets incremented on each async request start event, and decrements on each async complete event (doesn't seem thread safe to me). When the counter is zero a property bound to the UI turns the dialog off. Is there a better/more general way of dealing with this problem than my counter solution?
Your counter solution is a valid one. Whatever you do, you will have to keep track of all your requests and understand when they arrive (when count hits zero).
You can do different things to clean up your code like put all of this implementation in some MultiAsyncWaiter class which returns an event when complete. But the fundamental implmentation will remain the same: keep track of them until they all return.
You are right about the thread unsafe-ness of the int. If you use interlocked operations (see comments) or lock on the variable, you can keep your implementation thread safe.
Why volatile keyword wont work: With multiple threads changing the variable, an interlocked operation is required for the decrement, which is technically a read + write operation. This is because another thread can change the value between the read and the write.
I've been trying to work out how to cancel a long-running AD search in System.DirectoryServices.Protocols. Can anyone help?
I've looked at the supportControl/supportedCapabilities attributes on RootDSE and they don't contain the 1.3.6.1.1.8 OID so I think that means it doesn't support the LDAP CANCEL extended operation as defined here: https://www.rfc-editor.org/rfc/rfc3909
That leaves the original LDAP ABANDON command (see here for list). But there doesn't seem to be a matching DirectoryRequest Class.
Anyone have any ideas?
I think I've found my answer: whilst I was reading around your suggestion, Martin, I came across the Abort method on the LdapConnection class. I didn't expect to find it there: starting out from the LDAP documentation I'd expected to find it as just another LDAPMessage but the MS guys seem to have treated it as a special case. If anyone is familiar with a non-MS implementation of LDAP and can comment on whether the MS approach is typical, I'd appreciate it to improve my understanding.
I think, but I'm not positive, there is no asynch query with a cancel. It has an asynch property but it's to allow a collection to be filled, nothing to do with cancelling. The best I can offer is to put your query in a background worker thread and put an asynch callback that will deal with the answer when it comes back. If the user decides to cancel, you can just cancel the background worker thread. You'll free your app up, even if you haven't freed the ldap server up until it finishes it's query. You can find info on background worker threads at http://www.c-sharpcorner.com/UploadFile/LivMic/BGWorker07032007000515AM/BGWorker.aspx
Don't forget to call .Dispose() when cleaning up your active directory objects to prevent memory leaks.
If the query will produce many data also, you can abandon them through paging. Specify a PageResultRequestControl option in the query, giving a fairly low page size (IIUC, 1000 is the default page size). IIUC, you'll send new requests every time you got a page (passing cookies from one response into the next request). When you choose to cancel the query, send another request with zero expected results.