I am trying to build a data model for weekly service availability of users in the Firebase Firestore to store and query the data efficiently. Is there any known alternative method for Firestore for this kind of requirement? How can I improve this system if there are no better methods?
An arbitary possible entry by user:
Monday
Tuesday
1:00pm-3:00pm 5$ per hour
9:00am-9:45am 7$ per hour
10:00pm-11:45pm 10$ per hour
4:00pm-5:30pm $20 whole service
8:00pm-10:00pm $11 per hour
10pm-11:30pm $15 per hour
User should also be able to override the usual schedule on specific date for example:
30/01/2023
1:00pm 4:00pm 7$ per hour
When a service is booked from the user shouldn't be shown available on the booked time interval.
The data also should be a queryable combination of these:
Available users between specific time ranges.
Service price in a specific range (example: price <= 10 & price >= 3)
Price type (per hour or whole service)
January 5 is the first Monday of timestamp dates.
January 6 for Tuesday.
January 7 for Wednesday.
.
.
.
January 11 for Sunday.
Under the SchedulePlan/ collection a document looks like:
{
Thursday: [],
Friday: [],
Tuesday: [{start: January 6, 1970 a...],
Sunday: [], Wednesday: [{end: January 7, 1970 at ...],
Saturday: [],
Monday: [{start: January 5, 1970 a...]}
Schedule objects stored in the arrays of weekday or a override day:
{start: Timestamp, end: Timestamp, price: Number, priceType: String}
Date overrides are also arrayed and contain the specific date time range in the schedule object rather than the dates between Jan 5 and Jan 11.
I had to create a separate collection for querying the data since some array operations and inequality in multiple fields are not allowed or too limited.
a document for a Schedule object between 11:00pm:11:59pm-Thursday in the SearchIndex/ collection:
{{2315: true, 2330: true, 2345: true, 2359: true, Monday: true},
profileDuplicate{...userinfo...tags of user...},
schedulePlanId: "document id that this document generated from"}
In summary:
There are two collection that handles scheduling in this system if we ignore the profile of the users (on update of the profile related SearchIndex/ documents should be update to maintain consistency):
SchedulePlans/
SearchIndex/
SchedulePlans/ stores the schedule information in arrays. There is a document for a plan.
SearchIndex/ stores each scheduled time interval on a separate document so it is possible to query and list available users for any time interval.
For example, if we have schedule plan that only available Monday with 2 scheduled time intervals:
SearchIndex/
->1:00pm-3:00pm
->10:00pm-10:45pm
I hope the current system design is explained clearly.
If I try to query with where chain with only equalities(==) there are no problems but if I try to get available users in a time interval and with a specific price range composite indexing required.
How should composite indexing on SearchIndex/ be done? Should I add all combinations like where
Monday = true, price ASC
Monday = true, 0000 = true, price ASC
Monday = true, 0000 = true, 0015 = true, price ASC
Monday = true, 0000 = true, 0015 = true, 0030 = true, price ASC
... for all 24 * 4 = 96 and *7 for each schedule day. This seems too much.
OR
Monday = true, price ASC
Monday = true, 0000 = true, price ASC
Monday = true, 0015 = true, price ASC
composite indexes will work for:
Monday = true, 0000 = true, 0015 = true, price ASC
If it does, how can I modify the composite indexes to work with date overrides?
Is this model efficient enough or is there a better approach that will be able to query all the data at the minimum cost document read and write wise? Note that in this model if the user profile updates or schedule plan updates all indexes should updates accordingly.
Related
I am creating a report in Google Data Studio which pulls numerous days-worth of data; however, I want the report's date filter to default the initial presentation data to ONLY display the previous BUSINESS DAY's data.
I can set the default date range to "Yesterday" (or today minus 1) using the Advanced option. That get's me part of the way; however, the overnight, batch data I received each is based on activity conducted the prior weekday (Monday-Friday).
So, a date range of "Yesterday" (or today minus 1) works find when my report is executed on a Tuesday through Saturday, but if it is executed on a Sunday or a Monday, no data will show.
Is there a way Data Studio can handle this scenario? Essentially, I need "Yesterday" to ignore weekend days of Saturday and Sunday.
So I got this answer from Google Support
Click the pencil icon to edit your data source
Duplicate your date field and change the type to Day of Week
Add a control (type Drop-down list) to your dashboard.
Set Date range dimension to Date
Set Dimension to Day of Week and default selection 1,2,3,4,5.
You can do something like this:
CASE
WHEN (date_field = DATETIME_SUB(current_date(), INTERVAL 4 day) AND weekday(current_date()) = 1) THEN DATE(DATETIME_SUB(current_date(), INTERVAL 2 day))
WHEN (date_field = DATETIME_SUB(current_date(), INTERVAL 3 day) AND weekday(current_date()) = 1) THEN DATE(DATETIME_SUB(current_date(), INTERVAL 1 day))
WHEN (date_field = DATETIME_SUB(current_date(), INTERVAL 1 day) AND weekday(current_date()) = 2) THEN date_field
WHEN (date_field = DATETIME_SUB(current_date(), INTERVAL 3 day) AND weekday(current_date()) = 2) THEN DATE(DATETIME_SUB(current_date(), INTERVAL 2 day))
ELSE null
END
and then use that as your date range dimension. It basically makes Fridays and Thursdays look like yesterday and the day before on Monday and Friday the day before for Tuesday so you can use custom data ranges with the yesterday and previous period filters
We have a property booking site (similar to Airbnb) with a LAMP setup and a relatively simple SOLR index. A user can search by:
Location
Guests
Date IN/OUT
They can also filter results for a specific price range, as well as sort by price.
The issue is that price per night is never fixed, and is depends on several things such as custom prices (for one day, or a range of days), price per guest, discounts (e.g. weekend discount) etc. One example could be that price_per_night might be 50$, however for the 10th and 11th of August the real price per night could be 110$ plus 20$ extra guest fee, minus a discount of 30$ for that particular weekend. On top of this, commissions are applied on the total amount so we'd like this to be as accurate as possible.
While we can calculate this on the server side, this of course affects the results returned for a price range as well as the sorting (highest price first vs. lowest).
Could anyone suggest any possible solution?
Below is an example of a select:
"docs": [
{
"id": 1,
"property_type": 1,
"room_type": 1,
"minimum_nights": 2,
"maximum_nights": 120,
"location": "41.3902359,2.1685901",
"price_per_night": 210,
........
"unavailable_days": [
"2016-09-15T00:00:00Z",
"2016-09-16T00:00:00Z",
......
]
}
How I can set GAE cron job to run at specific date at specific time
Like 10th April at 12:20 minute.Please provide syntax for this use case.
How to set IST time zone.
From the cron format documentation:
If you want more specific timing, you can specify the schedule as:
("every"|ordinal) (days) ["of" (monthspec)] (time)
Where:
ordinal specifies a comma separated list of "1st", "first" and so
forth (both forms are ok) days specifies a comma separated list of
days of the week (for example, "mon", "tuesday", with both short and
long forms being accepted); "every day" is equivalent to "every
mon,tue,wed,thu,fri,sat,sun" monthspec specifies a comma separated
list of month names (for example, "jan", "march", "sep"). If omitted,
implies every month. You can also say "month" to mean every month, as
in "1,8,15,22 of month 09:00". time specifies the time of day, as
HH:MM in 24 hour time.
So you'd want something like:
schedule: 10 of april 12:20
timezone: Asia/Kolkata
Possible solutions:
1) Create a cronjob that runs once a minute. When the current time equals your desired time, run your code.
2) If the specific time is in the next 30 days, use a Task with the eta property set: https://developers.google.com/appengine/docs/python/taskqueue/tasks#Task
3) Use some external service to setup a webhook that gets called at the proper time, make your code run when the webhook is called.
I've been struggle with this for a while and hope someone can give me an idea to tackle this.
We have a service that goes out and collects Facebook likes, comments, and shares for each status update multiple times a day. The table that stores this data is something like this:
PostId EngagementTypeId Value CollectedDate
100 1(for likes) 10 1/1/2013 1:00
100 2 (comments) 2 1/1/2013 1:00
100 3 0. 1/1/2013 1:00
100. 1. 12 1/1/2013 3:00
100. 2. 3. 1/1/2013 3:00
100. 3 5. 1/1/2013 3:00
Value holds the total for each engagement type at the time of collection.
I got a requirement to create a report that shows new value per day at different time zones.
Currently,I'm doing the calculation in a stored procedure that takes in a time zone offset and based on that I calculate the delta for each day. If this is for someone in California, the report will show 12 likes, 3 comments, and 5 shares for 12/31/2012. But if someone with the time zone offset of -1, he will see 10 likes on 12/31/2012 and 2 likes on 1/1/2013.
The problem I'm having is doing the calculation on the fly can be slow if we have a lot of data and a big date range. We're talking about having the delta pre-calculated for each day and stored in a table and I can just query from that ( we're considering SSAS but that's for the next phase). But doing this, I would need to have the data for each day for 24 time zones. Am I correct (and if so, this is not ideal) or is there a better way to approach this?
I'm using SQL 2012.
Thank you!
You need to convert UTC DateTime stored in your column to Date based on users UTC time. This way you don't have to worry about any table that has to be populated with data. To get users date from your UTC column you will use something like this
SELECT CONVERT(DATE,(DATEADD(mi, DATEDIFF(mi, GETUTCDATE(), GETDATE()), '01/29/2014 04:00')))
AS MyLocalDate
The select statement above figures out Local date based on the difference of UTC date and local Date. You will need to replace GETDATE() with users DATETIME that is passed in to your procedure and replace '01/29/2014 04:00' with your column. This way when you select any date from your table it will be according to what that date was at users local time. Than you can calculate other fields accordingly.
I use a DATE's master table for looking up dates and other values in order to control several events, intervals and calculations within my app. It has rows for every single day begining from 01/01/1990 to 12/31/2041.
One example of how I use this lookup table is:
A customer pawned an item on: JAN-31-2010
Customer returns on MAY-03-2010 to make an interest pymt to avoid forfeiting the item.
If he pays 1 months interest, the employee enters a "1" and the app looks-up the pawn
date (JAN-31-2010) in date master table and puts FEB-28-2010 in the applicable interest
pymt date. FEB-28 is returned because FEB-31's dont exist! If 2010 were a leap-year, it
would've returned FEB-29.
If customer pays 2 months, MAR-31-2010 is returned. 3 months, APR-30... If customer
pays more than 3 months or another period not covered by the date lookup table,
employee manually enters the applicable date.
Here's what the date lookup table looks like:
{ Copyright 1990:2010, Frank Computer, Inc. }
{ DBDATE=YMD4- (correctly sorted for faster lookup) }
CREATE TABLE datemast
(
dm_lookup DATE, {lookup col used for obtaining values below}
dm_workday CHAR(2), {NULL=Normal Working Date,}
{NW=National Holiday(Working Date),}
{NN=National Holiday(Non-Working Date),}
{NH=National Holiday(Half-Day Working Date),}
{CN=Company Proclamated(Non-Working Date),}
{CH=Company Proclamated(Half-Day Working Date)}
{several other columns omitted}
dm_description CHAR(30), {NULL, holiday description or any comments}
dm_day_num SMALLINT, {number of elapsed days since begining of year}
dm_days_left SMALLINT, (number of remaining days until end of year}
dm_plus1_mth DATE, {plus 1 month from lookup date}
dm_plus2_mth DATE, {plus 2 months from lookup date}
dm_plus3_mth DATE, {plus 3 months from lookup date}
dm_fy_begins DATE, {fiscal year begins on for lookup date}
dm_fy_ends DATE, {fiscal year ends on for lookup date}
dm_qtr_begins DATE, {quarter begins on for lookup date}
dm_qtr_ends DATE, {quarter ends on for lookup date}
dm_mth_begins DATE, {month begins on for lookup date}
dm_mth_ends DATE, {month ends on for lookup date}
dm_wk_begins DATE, {week begins on for lookup date}
dm_wk_ends DATE, {week ends on for lookup date}
{several other columns omitted}
)
IN "S:\PAWNSHOP.DBS\DATEMAST";
Is there a better way of doing this or is it a cool method?
This is a reasonable way of doing things. If you look into data warehousing, you'll find that those systems often use a similar system for the time fact table. Since there are less than 20K rows in the fifty-year span you're using, there isn't a huge amount of data.
There's an assumption that the storage gives better performance than doing the computations; that most certainly isn't clear cut since the computations are not that hard (though neither are they trivial) and any disk access is very slow in computational terms. However, the convenience of having the information in one table may be sufficient to warrant having to keep track of an appropriate method for each of the computed values stored in the table.
It depends on which database you are using. SQL Server has horrible support for temporal data and I almost always end up using a date fact table there. But databases like Oracle, Postgres and DB2 have really good support and it is typically more efficient to calculate dates on the fly for OLTP applications.
For instance, Oracle has a last_day() function to get the last day of a month and an add_months() function to, well, add months. Typically in Oracle I'll use a pipelined function that takes start and end dates and returns a nested table of dates.
The cool way of generating a rowset of dates in Oracle is to use the hierarchical query functionality, connect by. I have posted an example of this usage in another thread.
It gives a lot of flexibility without the PL/SQL overhead of a pipelined function.
OK, so I tested my app using 31 days/month to calculate interest rates & pawnshops are happy with it! Local Law prays as follows: From pawn or last int. pymt. date to 5 elapsed days, 5% interest on principal, 6 to 10 days = 10%, 11 to 15 days = 15%, and 16 days to 1 "month" = 20%.
So the interest table is now defined as follows:
NUMBER OF ELAPSED DAYS SINCE
PAWN DATE OR LAST INTEREST PYMT
FROM TO ACUMULATED
DAY DAY INTEREST
----- ---- ----------
0 5 5.00%
6 10 10.00%
11 15 15.00%
16 31 20.00%
32 36 25.00%
37 41 30.00%
42 46 35.00%
47 62 40.00%
[... until day 90 (forfeiture allowed)]
from day 91 to 999, daily prorate based on 20%/month.
Did something bad happen in the UK on MAR-13 or SEP-1752?