We have a class CBSTV, which has some subclasses like News, Documentary, and etc. The data property time will specify the time of each program. For example, the news is every day between 16:00:00-17:00:00. I tried to represent this restriction this way:
dateTime [>= T16:00:00Z, <=T17:00:00Z ]
but, it is wrong, anyone know the correct way to represent this
also, we have another data property duration. For instance the duration of the news is between 45 minutes to one hour. I don't know how I should represent this one too?
The datatype for recurring times (e.g., 17:00 every day) is not xsd:dateTime but xsd:time
Related
Need to get entities filtering by month instead of complete date values (E.g. Birthdays) using Google App Engine Text Search. On verifying GAE docs, I think it is not possible to query date fields by month directly.
So in order to filter them by month/date, we consider saving each date sub value like Date(DD), Month(MM) and Year(YYYY) as separate NUMBER field along with complete date field.
I verified locally that we can achieve by saving like this. But is this the correct way of saving dates by splitting each field when we want to query on date sub values?
Is there any known/unknown limit on number of fields per document apart from 10GB size limit in GAE Text Search?
Please suggest me.
Thanks,
Naresh
The only time NUMBER or DATE fields make sense is if you need to query on ranges of values. In other cases they are wasteful.
I can't tell from your question exactly what queries you want to run. Are you looking for a (single) specific day of the month (e.g., January 6 -- of any year)? Or just "anything in June (again, without regard to year)"? Or is it a date range: something like January 20 through February 19? Or July 1 through September 30?
If it's a range then NUMBER values may make sense. But if it's just a single specific month, or a single month and day-of-month combination, then you're better off storing month and day as separate ATOM fields.
Anything that looks like a number, but isn't really going to be searched via a numerical range, or done arithmetic on, isn't really a number, and is probably best stored as an ATOM. For example, phone numbers, zip codes (unless you're terribly clever and wanting to do something like "all zip codes in San Francisco look like 941xx" -- but even then if that's what you want to do, you're probably better off just storing the "941" prefix as an ATOM).
I am looking for some help as to the best way to structure data in app engine ndb using python, process it and query it later. I want to store temperature data at hourly intervals for different geographical regions.
I can think of two entity options but there maybe something much better. The first would be to store the hourly temperature in individual properties:
class TempData(ndb.Model):
region = ndb.StringProperty()
date = ndb.DateProperty()
00:00 = ndb.FloatProperty()
01:00 = ndb.FloatProperty()
...
23:00 = ndb.FloatProperty()
Or I could store the data
class TempData(ndb.Model):
region = ndb.StringProperty()
date = ndb.DateProperty()
time = ndb.TimeProperty()
temp = ndb.FloatProperty()
(it might be better to store date and time as one property?)
I want to be able to query the datastore to calculate the Total, Max, Min, and average temperature for any given date range. In the first option I could potentially create 4 more properties to effectively pre-process and store the Total, Max etc for each day so if I wanted to query the total temperature for a year I would only have to sum 365 values as opposed to 8760? I'm not sure how I would do this in the second option?
I am relatively new to app engine and datastore and I think I am still thinking in terms of relationship db's so any help would really be appreciated. Later on it might be necessary to store data in different time zones.
Thanks
Paul
Personally, I'd go with a variant of the first approach:
class TempData(ndb.Model):
region = ndb.StringProperty()
date = ndb.DateProperty()
temp = ndb.FloatProperty(repeated=True)
using the temp list to store temperatures by hour in order as you learn about them. I don't think the preprocessing per-date will add anything much: to compute whatever for a year, you'd still need to fetch 365 entities, and the delay for that will swamp the tiny amount of time required to sum up a few thousand numbers anyway.
In general, preprocessing is useful if you want to handily query by the new fields you create by such processing (e.g rapidly answer the question "which dates in locale X had average temperatures greater than 20 Celsius"). That does not seem to be your use case.
If anything, if it's common for you to have to compute many-month values, preprocessing to aggregate things per-month (into simpler TempDataMonth entities) may be more useful. Or, any other several-days period you find useful, of course (weeks, ten-day-groups, whatever). Those could be computed in a background task periodically checking which such periods have become complete since the last check. But, this is a bit beyond your question, so I'm not getting into fine-grained details.
The general idea is that minimizing the number of entities to fetch tends to be the single most important optimization; other optimizations are of course also possible, but, they tend to play second fiddle to that:-).
What am I doing wrong in this query?
SELECT * FROM TreatmentPlanDetails
WHERE
accountId = 'ag5zfmRvbW9kZW50d2ViMnIRCxIIQWNjb3VudHMYtcjdAQw' AND
status = 'done' AND
category = 'chirurgia orale' AND
setDoneCalendarEventStartTimestamp >= [timestamp for 6 june 2012] AND
setDoneCalendarEventStartTimestamp <= [timestamp for 11 june 2012] AND
deleteStatus = 'notDeleted'
ORDER BY setDoneCalendarEventStartTimestamp ASC
I am not getting any record and I am sure there are records meeting the where clause conditions. To get the correct records I have to widen the timestamp interval by 1 millisecond. Is it normal? Furthermore, if I modify this query by removing the category filter, I am getting the correct results. This is definitely weird.
I also asked on google groups, but I got no answer. Anyway, for details:
https://groups.google.com/forum/?fromgroups#!searchin/google-appengine/query/google-appengine/ixPIvmhCS3g/d4OP91yTkrEJ
Let's talk specifically about creating timestamps to go into the query. What code are you using to create the timestamp record? Apparently that's important, because fuzzing with it a little bit affects the query. It may be relevant that in the datastore, timestamps are recorded as integers representing posix timestamps with microseconds, i.e. the number of microseconds since 1/1/1970 UTC (not counting leap seconds). It's also relevant that dates (i.e. without a time) are represented as midnight, i.e. the earliest time on that day. But please show us the exact code. (It may also be important to show the actual content of the record that you're attempting to retrieve.)
An aside that is not specific to your question: Entity property names count as part of your storage quota. If this is going to be a huge dataset, you might pay more $$ than you'd like for property names like setDoneCalendarEventStartTimestamp.
Because you write :
if I modify this query by removing the category filter, I am getting
the correct results
this probably means that the category was not indexed at the time you write the matching records to the data store. You have to re-write your records to the data store if you want them added to the newly created index.
I have a WPF app, where one of the fields has a numeric input box for length of a phone call, called ActivityDuration.
Previously this has been saved as an Integer value that respresents minutes. However, the client now wishes to record meetings using the same table, but meetings can last for 4-5 hours so entering 240 minutes doesn't seem very user friendly.
I'm currently considering my options, whether to change ActivityDuration to a time value in SQL 2008 and try to use a time mask input box, or keep it as an integer and present the client with 2 numeric input boxes, one for hours and one for minutes and then do the calculation to save it in SQL Server 2008 as integer minutes.
I'm open to comments and suggestions. One further consideration is that I will need to be able to calculate total time based upon the ActivityDuration so the field DataType should allow it to be summed easy.
The new time datatype only supports 24 hours, so if you need more you'll have to use datetime.
So if sum 7 x 4 hour meetings, you'll get "4 hours" back
How the DB stores it is also different to how you present and capture the data.
Why not hh:nn type display and convert in the client and store as datetime?
Track the start and end time, no need to mask out the date, since the duration will just be a calculation off of the two dates. You can even do this in "sessions" such that one meeting can have multiple sessions (i.e. one meeting that spans across lunch, that shouldn't be counted toward the duration...).
The data type, then is either datetime or smalldatetime.
Then to get the "total duration" it's just a query using
Select sum(datediff(mm, startdate, enddate)) from table where meetingID = 1
I am making a small app that plots financial price data and since the finance markets are closed on the weekends, I have no data for those days. By default the chart, found in the new WPF Toolkit, shows a large gap between Friday and following Monday and this behaviour is not acceptable. I am trying to figure out a way to "hide" the weekend gaps but can't seem to find any good solutions. So far, I figured that I would have to overload the standard DateTime struct (how?) which will be able to recognize and skip weekends and holidays. I am looking for suggestions and/or pointers before I start down that slippery slope.
Some more details:
I am given a wide range of data - currently daily closing prices on NYSE. I am using the DayTimeAxis to plot the independent variable of LineSeries which is of type DateTime. It currently simply plots all the prices, one day at a time - and that's where the devil is, it shows wider gaps due to lack of data for Saturdays and Sundays and some major holidays.
I will eventually have to show more detailed (hourly, minute) chart once data becomes available, but the problem will remain if the user will want to view hourly data for some Friday and the following Monday.
After much playing around with various options, I ended up using the CategoryAxis instead of DateTimeAxis. It treats each day as a category without inferring the relationship between the days.
The data can still be kept as DateTime objects for any necessary calculations and you just have to worry about the ToString (that's what CategoryAxis to label each category). Or just throw them all in as strings - technically calculations are done on the data points, not the array of dates so not much of a loss here...
If you will plot candlestick or ohlc you can't go with toolkit (you can but it will be VERY unpleasant) if you are open to use component use visifire's the one you may go. if you can't figure it out with that send a sample code please.
Completely off the top of my head, so I'm not sure how feasible this would be, but could you possibly set up a value converter on the Width property of the data point that would return "Auto" on a weekday and 0 for a weekend?