I'm using ibatis and its cache model for the first time and i cant seem to figure out at what time will the cache be filled with new results, meaning whne will the query that i'm applying cache model on be executed again to fill the cache with new results.
Does it require some sort of command to get the results again or will it be done automatically? any help will be appreciated.
Thanks
It will be filled again when it is flushed which is done over events such as fluchoninterval, flushOnupdate, flushoninsert.
along with the above answer, you can also define flush intervals. You can define the flush interval in hours, mins & seconds and let the flush happening every x hours, y mins, z seconds interval.
Related
I have an AppleScript that runs on loop every two hours to modify a calendar B based on updates from another calendar A.
The script uses the on idle command below to wait 2 hours every loop. What happens if the computer stays idle for 1.5 hours then goes to sleep for 10 hours? Will there be 0.5 hours left when it wakes up? Any other scenarios?
on idle
my_code()
return (120 * minutes)
end idle
The script truly only needs to run if there is an update to calendar A, which is a shared iCloud calendar and can get updates from multiple people. The two hour loop is what I could figure out so far but I feel it is not efficient. Any more robust suggestions? Is there a way I can trigger the script to run only when it detects an update in calendar A? Or, along the same line of thought, is there a way to get the last timestamp the calendar was updated?
Thanks
I can't test following. Not sure it is the best way to solve your problem. Try yourself:
property oldStampDates : {}
on run
tell application "Calendar" to tell calendar "Test Calendar" to set oldStampDates to get stamp date of events
end run
on idle
--> Script retrieves last modified date and time of indicated calendar events.
tell application "Calendar" to tell calendar "Test Calendar" to set newStampDates to get stamp date of events
if newStampDates is not oldStampDates then display notification "The changes was detected"
set oldStampDates to newStampDates
return 30 -- seconds, default setting
end idle
NOTE: 1) you can put instead of display notification call to your handler my_code(), 2) you can put instead of 30 seconds other value, for example, return 10 (checking every 10 seconds).
We have an API that queries an Influx database and a report functionality was implemented so the user can query data using a start and end date.
The problem is that when a longer period is chosen(usually more than 8 weeks), we get a timeout from influx, query takes around 13 seconds to run. When the query returns a dataset successfully, we store that in cache.
The most time-consuming part of the query is probably comparison and averages we do, something like this:
SELECT mean("value") AS "mean", min("value") AS "min", max("value") AS "max"
FROM $MEASUREMENT
WHERE time >= $startDate AND time < $endDate
AND ("field" = 'myFieldValue' )
GROUP BY "tagname"
What would be the best approach to fix this? I can of course limit the amount of weeks the user can choose, but I guess that's not the ideal fix.
How would you approach this? Increase timeout? Batch query? Any database optimization to be able to run this faster?
In such cases where you allow user to select in days, I would suggest to have another table that stores the result (min, max and avg) of each day as a document. This table can be populated using some job after end of the day.
You can also think changing the document per day to per week or per month, based on how you plot the values. You can also add more fields like in your case, tagname and other fields.
Reason why this is superior to using a cache: When you use a cache, you can store the result of the query, so you have to compute for every different combination in realtime. However, in this case, the cumulative results are already available with much smaller dataset to compute.
Based on your query, I assume you are using InfluxDB v1.X. You could try Continuous Queries which are InfluxQL queries that run automatically and periodically on realtime data and store query results in a specified measurement.
In your case, for each report, you could generate a CQ and let your users to query it.
e.g.:
Step 1: create a CQ
CREATE CONTINUOUS QUERY "cq_basic_rp" ON "db"
BEGIN
SELECT mean("value") AS "mean", min("value") AS "min", max("value") AS "max"
INTO "mean_min_max"
FROM $MEASUREMENT
WHERE "field" = 'myFieldValue' // note that the time filter is not here
GROUP BY time(1h), "tagname" // here you can define the job interval
END
Step 2: Query against that CQ
SELECT * FROM "mean_min_max"
WHERE time >= $startDate AND time < $endDate // here you can pass the user's time filter
Since you already ask InfluxDB to run these aggregates continuously based on the specified interval, you should be able to trade space for time.
I have a bunch of data to monitor. My data are statistics that can only be retrieved every hour but can change every second and I want to store into a database as much values as I can for each data set.
I've though about several approaches for this problem and I finally chose to refresh and read all statistics at once instead of reading them independently.
So that, I came out with command mycommand which reads all my statics with the cost of several minutes (let's say 30) of execution. Now I would like to run this script every hour, but taking the script execution into account.
I actually run
* */1 * * * mycommand.sh
and receive many annoying error emails (actually one every hour) and I effectly retrieve my statistics every 2 hours.
1h 30 minutes is the half of 3 hours. So you could have two entries in crontab(5) running the same /home/gogaz/mycommand.sh script, one to run it at 1, 4, 7, ... hours (every 3 hours from 1am) and another to run it at 2:30, 5:30, 8:30 hours, ... (every 3 hours from 2:30am) etc
Writing these entries is left as an exercise to the reader.
See also anacrontab(5) and at(1). For example, you might run your script once using batch, but terminate your script with an at command rescheduling that same script (the drawback is handling of unexpected errors).
If you redirect your stdout and stderr in your crontab entry, you won't get any emails.
My program fetches ~100 entries in a loop. All entries are fetched using get_by_key_name(). Appstats show that some get_by_key_name() requests are taking as much as 750ms! (other big values are 355ms, 260ms, 230ms). Average for other fetches ranges from 30ms to 100ms. These times are in real_time and hence contribute towards 'ms' and not 'cpu_ms'.
Due to the above, total time taken to return the webpage is very high ms=5754, where cpu_ms=1472. (above times are seen repeatedly for back to back requests.)
Environment: Python 2.7, webapp2, jinja2, High Replication, No other concurrent requests to the server, Frontend Instance Class is F1, No memcache set yet, max idle instances is automatic, min pending latency is automatic, using db (NOT NDB).
Any help will be greatly appreciated as I based whole database design on fetching entries from the datastore using only get_by_key_name()!!
Update:
I tried profiling using time.clock() before and immediately after every get_by_key_name() method call. The difference I get from time.clock() for every single call is 10ms! (Just want to clarify that the get_by_key_name() is called on different Kinds).
According to time.clock() the total execution time (in wall-clock time) is 660ms. But the real-time is 5754 (=ms), and cpu_ms is 1472 per GAE logs.
Summary of Questions:
*[Update: This was addressed by passing list of keys] Why get_by_key_name() is taking that long?*
Why ms of 5754 is so much more than cpu_ms of 1472. Is task execution in halted/waiting-state for 75% (1-1472/5754) of the time due to which real-time (wall clock) time taken is so long as far as end user is concerned?
If the above is true, then why time.clock() shows that only 660ms (wall-clock time) elapsed between start of the first get_by_key_name() request and the last (~100th) get_by_key_name() request; although GAE shows this time as 5754ms?
SET a 100
EXPIRE a 86400
OK, right now, "a" will expire within a day. I want SET "a" to other values, but I still want it to expire based on the initial "EXPIRE" that I created.
In other words, don't remove the EXPIRE when I set "a" to something new.
This is something I requested 2 years ago and now it was added (https://github.com/antirez/redis/pull/6679).
SET foo 1 KEEPTTL
This will get in with some next release.
If you're doing this from the command line, you can't do it precisely. The best you could do is to manually set the new key's expire time as a's remaining TTL (TTL a).
If you're communicating with Redis from a program you could probably fetch a's expire time and set that value as the new key's expire time fast enough to be precise.
try setex bcz its faster & atomic