I'm currently using the Google memcache API in my Google Appsengine application to store a large amount of data in cache, however this needs to happen asynchronously as I need to return a result before it is done.
I already found an answer here stating how it is done, however I still can't get my code to work.
I tried using this, however it simply causes memcache to be store the values synchronously:
client = memcache.Client()
rpc = client.set_multi_async(values)
rpc.get_result()
return values[id]
I also tried this, but it causes memcache to never save the values:
client = memcache.Client()
client.set_multi_async(values, rpc=memcache.create_rpc())
return values[id]
Is there any way to store the values asynchronously and return a value at the same time? Thanks
rpc.get_result() causes the rpc object to wait until the result is available before returning, thus making your code synchronous. In order to make your code asynchronous, you would need to return the rpc object(s) and then use the Future class to wait for and handle results when they are ready.
Related
I'm trying cacheing query, condition is after first request and if query object is not empty I want to cache. I have tried like below
public function getCategories()
{
$categoriesTable = TableRegistry::getTableLocator()->get( 'Categories' );
if(\Cake\Cache\Cache::read('categories', 'redis_cache') == null)
{
$categories = $categoriesTable->find();
if(!$categories->isEmpty())
{
print("hello");
$categories->cache('categories','redis_cache');
debug(\Cake\Cache\Cache::read('categories', 'redis_cache'));
}
$this->set('categories',$categories);
}else{
$this->set('categories',\Cake\Cache\Cache::read('categories', 'redis_cache'));
}
$this->viewBuilder()->setOption('serialize', ['categories']);
}
Output I got in postman
1st time hit
hello
null
2nd time hit
hello
null
But after comment if(!$categories->isEmpty()) condition
1st time hit
{
"categories": [
{
"id": 2,
...
}
Also if I write the condition like below it's also working fine.
if(!$categoriesTable->find('all')->isEmpty()). // it's working
What's the wrong I'm doing here ?
Queries are lazy evaluated, meaning they are not being executed until the data is actually being requested, however isEmpty() is one of the methods that will cause the query to be executed, it needs the results to tell whether there are any.
Your code sets the query cacher config afterwards, which will have no effect, as further evaluation/iteration of the query object will not execute the query again (which is when it would normally cache its results), but access the runtime buffered results instead, hence no data is being cached.
If you need more control over how caching is being handled, then you maybe want to avoid the built-in query caching, and do it completely manually, eg read and also write the cache as neccessary.
As far as your use case is concerned, if you have a script running that deletes and regenerates the data, you might want to consider using it to also clear the cache afterwards (bin/cake cache clear redis_cache). Then, if you implement the "try again until there's data" logic in the frontend, your backend code can be simplified to just read and cache whatever data there is, and if there is none yet, your frontend will let the user know that the data isn't ready yet and poll again. While websockets or server sent events might be better, long polling is certainly still preferable over having a script run minutes on end.
guild.members.fetch()
Returns a cached response. Is there a way to determine how long it takes for the cache to refresh?
Is there a fixed amount of time that passes between regular refreshes?
I am aware that the call returns a promise and must be either awaited or handled with .then(). I was actually wondering about the returned result of the call; how often discord clears it's own cache.
Thanks.
That function returns a promise, which resolves when all the members are fetched successfully and the cache is set. Simply await this promise to get updated cache.
// note this returns a Collection of the members who were fetched and cached
await guild.members.fetch()
console.log(guild.members.cache.size) // should log the real amount of members in the server
When I run a .fetch() command, it first returns null, then say suppose I have 100 documents and it will keep on loading from 1 to 100 and the counter keeps updating from 1 to 100 progressively. I don't want that to happen. I want all the results to be displayed at once after the fetch process has been completed.
Also, how can I display a relevant message to the user if no documents exist? The fetch method doesn't work for me since it returns 0 at first and hence "No document found" flashes for a second.
dbName.find({userID:"234234"}).fetch()
Even though the above has 100 docs, it first shows null and then keep loading the documents one by one. I want it load all at once or just return something if no docs exist.
I don't want that to happen. I want all the results to be displayed at once after the fetch process has been completed
To really obtain all documents at once on the client you will have to write a Meteor Method that returns all the documents:
Meteor.methods({
'allDocs' () {
return dbName.find({userID:"234234"}).fetch()
}
})
Note, that you have to call fetch on the cursor to return the documents, otherwise you will face an "unhandled promise rejection error".
Then call it from the client as usually. You can even add the documents to your client side local collection without affecting allow/deny (which should be off / deny all by default):
Meteor.call('allDocs', (err, documents) => {
// ... handle err
// all client collections have a local collection accessible via ._collection
const localCollection = dbName._collection
documents.forEach(doc => localCollection.insert(doc))
})
Advantages:
Returns all documents immediately
Less resources consumed (no publication observers required)
Works with caching tools, such as ground:db, to create offline-first applications
Disadvantages:
You should limit the query and access to your collections using Methods as much as possible, (using mdg:validated-method) which can require much more effort than shown in this examples
Not reactive! If you need reactivity on the client you need to include Tracker and reactive data-sources (ReactiveVar etc.) to provide some decent reactive user experience
Manual syncing can become frustrating and is error prone
Your question is actually about the subscription and it's state of readiness. While it is not yet ready, you can show a loading page, and once it is you can run the .fetch() to get the whole array. This logic could be put in your withTracker call, e.g.:
export default withTracker((props) => {
const sub = Meteor.subscribe('users');
return {
ready: sub.ready(),
users: sub.ready() && Users.find({userID: props.userID}).fetch()
};
})(UserComponent);
Then, in your component, you can decide whether to render a spinner (while ready == false), or the users.
Your question is not entirely clear to me in terms of tools etc (please state which database connector lib are you using), but firstly, given you're doing a database access, most likely, your ".fetch()" call is not a sync function but async, and also most likely, handled by a promise.
Secondly, given that you're using react, you want to set a new state only after you get all the results back.
If fetch is a promise then just do:
dbName.find({userID:"234234"}).fetch().then(results =>
setState({elements:results.data}) // do your processing accordingly
}
By only calling setState inside the promise, you'll always have all the results fetched at that instant and only with that do you update your component state with the setState function - either using your react component class this.setState or with hooks like useState (much cleaner).
If I call an async data store operation such as the one shown below but then end the request without calling get on the future, what will happen?
Will my operation still execute?
Will me response be sent before the operation has completed execution?
AsyncDatastoreService datastore = DatastoreServiceFactory.getAsyncDatastoreService();
Entity entity = new Employee("Employee", "Alfred");
// ... populate entity properties
// Make a sync call via the async interface
datastore.put(key)
//Return response
The rpc will be sent immediately; when your app is ready to send a response to the client, it will block until the rpc is done.
I've done this in python by accident and the result was nothing was written to the datastore.
Your operation may still execute but it seems that'll happen only if the response handler is still active when it decides to execute. If not, nothing seems to happen at all.
Yes, the response will be sent before the operation has completed execution - this is the main feature of a future, it's non-blocking.
I am calling an asynchronous service from my Silverlight app and I want to be able to cancel that call after it is made. There is an option for e.Cancelled once the service has finished (i.e. If e.Cancelled Then), but how to you set that cancelled to true after you have called it? How do you cancel that asynchronous call?
Let me clarify a bit... what I am trying to do is call the SAME method twice, one right after the other, and get the results of the last call into my collection. If I call an asynchronous method twice there is no guarantee that the second call will return first, so I may end up with the results of the first call coming in last and having the wrong results in my collection. So what I would like to do is cancel the first call when I make the second so I don't get results back from the first call. Seeing as how there is a Cancelled flag in the completed event args I figure you should be able to do this. But how?
It's async... the transfer is passed off to a remote server and it does not return until the server is done with it.
Basically the server will keep going, but you don't have to wait for the response. Disconnect your service completed event handler and pretend it was never called. That will give the effect of cancelling the operation.
If you really need to cancel something in progress on the server you would need to make another call to the server to cancel the first call. Assuming the first call is a very slow one, this might be possible.
Update (as question changed)
In the case you specify, it will be up to the server to cancel a operation in progress if a second one comes through, not up to the client. e.Cancelled is set server-side.
However... :)
You have exposed a client usability issue. Shouldn't you also delay sending any service request until an idle delay has passed. That way rapid selections will not result in multiple service calls.
Also... :>
You may also want to send a sequence number to your service calls and return that as part of the result. Then you will know if it is the latest request or not.
It sounds like what you really want to do is ignore the responses of all but the most recent call.
Set a unique ID (could be request #, a Guid, timestamp, or whatever) with the request, and make sure the service sends that same value back. Keep around the ID of the most recent request and ignore response that don't match that ID.
This will be safer than cancelling the first request, since if the service has already started sending the response before the cancel request happens, you still get your error condition.