I use a DATE's master table for looking up dates and other values in order to control several events, intervals and calculations within my app. It has rows for every single day begining from 01/01/1990 to 12/31/2041.
One example of how I use this lookup table is:
A customer pawned an item on: JAN-31-2010
Customer returns on MAY-03-2010 to make an interest pymt to avoid forfeiting the item.
If he pays 1 months interest, the employee enters a "1" and the app looks-up the pawn
date (JAN-31-2010) in date master table and puts FEB-28-2010 in the applicable interest
pymt date. FEB-28 is returned because FEB-31's dont exist! If 2010 were a leap-year, it
would've returned FEB-29.
If customer pays 2 months, MAR-31-2010 is returned. 3 months, APR-30... If customer
pays more than 3 months or another period not covered by the date lookup table,
employee manually enters the applicable date.
Here's what the date lookup table looks like:
{ Copyright 1990:2010, Frank Computer, Inc. }
{ DBDATE=YMD4- (correctly sorted for faster lookup) }
CREATE TABLE datemast
(
dm_lookup DATE, {lookup col used for obtaining values below}
dm_workday CHAR(2), {NULL=Normal Working Date,}
{NW=National Holiday(Working Date),}
{NN=National Holiday(Non-Working Date),}
{NH=National Holiday(Half-Day Working Date),}
{CN=Company Proclamated(Non-Working Date),}
{CH=Company Proclamated(Half-Day Working Date)}
{several other columns omitted}
dm_description CHAR(30), {NULL, holiday description or any comments}
dm_day_num SMALLINT, {number of elapsed days since begining of year}
dm_days_left SMALLINT, (number of remaining days until end of year}
dm_plus1_mth DATE, {plus 1 month from lookup date}
dm_plus2_mth DATE, {plus 2 months from lookup date}
dm_plus3_mth DATE, {plus 3 months from lookup date}
dm_fy_begins DATE, {fiscal year begins on for lookup date}
dm_fy_ends DATE, {fiscal year ends on for lookup date}
dm_qtr_begins DATE, {quarter begins on for lookup date}
dm_qtr_ends DATE, {quarter ends on for lookup date}
dm_mth_begins DATE, {month begins on for lookup date}
dm_mth_ends DATE, {month ends on for lookup date}
dm_wk_begins DATE, {week begins on for lookup date}
dm_wk_ends DATE, {week ends on for lookup date}
{several other columns omitted}
)
IN "S:\PAWNSHOP.DBS\DATEMAST";
Is there a better way of doing this or is it a cool method?
This is a reasonable way of doing things. If you look into data warehousing, you'll find that those systems often use a similar system for the time fact table. Since there are less than 20K rows in the fifty-year span you're using, there isn't a huge amount of data.
There's an assumption that the storage gives better performance than doing the computations; that most certainly isn't clear cut since the computations are not that hard (though neither are they trivial) and any disk access is very slow in computational terms. However, the convenience of having the information in one table may be sufficient to warrant having to keep track of an appropriate method for each of the computed values stored in the table.
It depends on which database you are using. SQL Server has horrible support for temporal data and I almost always end up using a date fact table there. But databases like Oracle, Postgres and DB2 have really good support and it is typically more efficient to calculate dates on the fly for OLTP applications.
For instance, Oracle has a last_day() function to get the last day of a month and an add_months() function to, well, add months. Typically in Oracle I'll use a pipelined function that takes start and end dates and returns a nested table of dates.
The cool way of generating a rowset of dates in Oracle is to use the hierarchical query functionality, connect by. I have posted an example of this usage in another thread.
It gives a lot of flexibility without the PL/SQL overhead of a pipelined function.
OK, so I tested my app using 31 days/month to calculate interest rates & pawnshops are happy with it! Local Law prays as follows: From pawn or last int. pymt. date to 5 elapsed days, 5% interest on principal, 6 to 10 days = 10%, 11 to 15 days = 15%, and 16 days to 1 "month" = 20%.
So the interest table is now defined as follows:
NUMBER OF ELAPSED DAYS SINCE
PAWN DATE OR LAST INTEREST PYMT
FROM TO ACUMULATED
DAY DAY INTEREST
----- ---- ----------
0 5 5.00%
6 10 10.00%
11 15 15.00%
16 31 20.00%
32 36 25.00%
37 41 30.00%
42 46 35.00%
47 62 40.00%
[... until day 90 (forfeiture allowed)]
from day 91 to 999, daily prorate based on 20%/month.
Did something bad happen in the UK on MAR-13 or SEP-1752?
Related
The History:
I have a data set that refreshes every Monday morning adding last week's values to a growing tally until there is 52 weeks in the data set (9 separate cohorts), across 38 different departments.
I have built a power query to filter the department and compiled tables for each cohort, limiting the data to the last 17 weeks, and using excel forecast modelling then setup each table to forecast 16 weeks ahead.
Because the week beginning (WB) dates keep changing IO cant hard code the result table to cells within each cohort table.
My result table needs to show current month, month +1, month +2, and month +3 forecast values as per the highest date closest to or equal to EOM and I need this to be automated, hence a formula.
PS added complexity is that the table has date/value adjacent in (last 17 weeks) and columns separated in future 16 weeks of data in each table. Structure is exactly the same across all the 9 cohort forecast tables.
My Question:
Am I best to use a nested EOM formula, or VLOOKUP(MAX) based on the cohort_forecast_table image link below?
Because the current month needs to be current I have created a cell using =NOW().
I then complete a VLOOKUP within each cell in the master table that references the references the data in each sub-table usin MAX and EOMONTH for current month, then month+1, month+2, month+3, etc.
In a simplified broken down solution:
Date array = 'D3:D35'
Volume array = 'E3:E35'
End of current month formula cell B3: =MAX(($D$3:$D$35<EOMONTH(D1,0))*D3:D35)
Call for result in cell C3:
'=VLOOKUP(B3,Dates:Volumes,2,FALSE)'
I think this will work for me and thank you all...
I need to calculate how much a plan has cost the customer in a specific month.
Plans have floating billing cycles of a month's length - for example a billing cycle can run from '2014-04-16' to '2014-05-16'.
I know the start date of a plan, and the end date can either be a specific date or NULL if the plan is still running.
If the end date is not null, then the customer is charged for a whole month - not pro rated. Example: The billing cycle is going from the 4th to 4th each month, but the customer ends his plan on the 10th, he will still be charged until the 4th next month.
Can anyone help me? I feel like I've been going over this a million times, and just can't figure it out.
Variables I have:
#planStartDate [Plan's start date]
#planEndDate [Plan's end date - can be null]
#billStartDate [The bill's start date - example: 2015-02-01]
#billEndDate [One month after bill's start date - 2015-03-01]
#price [the plan's price per billing cycle]
Heres the best answer I can give based on the very small information you have given so far(btw, in the future, it would really help people answer your question faster/easier/more efficiently if you could specify a lot more info;tables involved, all columns, etc..):
"I need to calculate how much a plan has cost the customer in a specific month."
SELECT SUM(price), customerID(I assume you have a column of some sort in this table to distinguish between customers) FROM table_foo
where planStartDate BETWEEN = 'a specific date you specify'
Its a bit rough of a query, but thats the best I can give till you specify more clearly your variable (i.e. tables involved, ALL columns in table, etc etc.....)
I've been struggle with this for a while and hope someone can give me an idea to tackle this.
We have a service that goes out and collects Facebook likes, comments, and shares for each status update multiple times a day. The table that stores this data is something like this:
PostId EngagementTypeId Value CollectedDate
100 1(for likes) 10 1/1/2013 1:00
100 2 (comments) 2 1/1/2013 1:00
100 3 0. 1/1/2013 1:00
100. 1. 12 1/1/2013 3:00
100. 2. 3. 1/1/2013 3:00
100. 3 5. 1/1/2013 3:00
Value holds the total for each engagement type at the time of collection.
I got a requirement to create a report that shows new value per day at different time zones.
Currently,I'm doing the calculation in a stored procedure that takes in a time zone offset and based on that I calculate the delta for each day. If this is for someone in California, the report will show 12 likes, 3 comments, and 5 shares for 12/31/2012. But if someone with the time zone offset of -1, he will see 10 likes on 12/31/2012 and 2 likes on 1/1/2013.
The problem I'm having is doing the calculation on the fly can be slow if we have a lot of data and a big date range. We're talking about having the delta pre-calculated for each day and stored in a table and I can just query from that ( we're considering SSAS but that's for the next phase). But doing this, I would need to have the data for each day for 24 time zones. Am I correct (and if so, this is not ideal) or is there a better way to approach this?
I'm using SQL 2012.
Thank you!
You need to convert UTC DateTime stored in your column to Date based on users UTC time. This way you don't have to worry about any table that has to be populated with data. To get users date from your UTC column you will use something like this
SELECT CONVERT(DATE,(DATEADD(mi, DATEDIFF(mi, GETUTCDATE(), GETDATE()), '01/29/2014 04:00')))
AS MyLocalDate
The select statement above figures out Local date based on the difference of UTC date and local Date. You will need to replace GETDATE() with users DATETIME that is passed in to your procedure and replace '01/29/2014 04:00' with your column. This way when you select any date from your table it will be according to what that date was at users local time. Than you can calculate other fields accordingly.
What would be the best way to store Quarter and Year in database? I have payments table and I need to assign quarter/year so that it's easy to tell for which quarter the payment was made.
I was thinking on:
a) adding two int columns to each payment
b) adding another table and add possible values up to 5 years ahead and use the ID to join that table with payments one.
What are other options? Maybe some is better or/and easier to maintain. This database will be used with C# program.
If you have to use separate year and quarter instead of a date (since you seem to have specific reporting requirements), I would go for a tinyint for quarter and smallint for year and store them in the PAYMENT table itself.
I would not store it in a different table. This is bad since:
You have to make sure you have produced enough years/quarters
You have to join and use a foreign key
If you store the data with the record, it will help performance on reads. Your table could be small but it is always good to keep in mind performance.
WHY
Let's imagine you need to get
all payments in specific quarter
where payment has been more than
specific amount and customer is a
particular customer
In this case, you would need a covering index on all items and still does not help since your query is for specific quarter and not quarter year. Having the data on the table, however, will help with lighter execution plan.
I've always just used datetime value with the 1st of January/April/July/October representing each quarter. Makes computation of the start/end dates of the quarter simple:
Start Date: the datetime column itself.
End Data: dateadd(month,3,quarterColumn)
Another alternative would be ISO 8601. Here's an ISO 8601 profile for use in Internet protocols: RFC 3339 (proposed standard).
An ISO 8601 representation of each quarter of the year 2011 looks like this:
2011-01-01/P3M
2011-04-01/P3M
2011-07-01/P3M
2011-10-01/P3M
The above specify a duration by starting date and duration (in this case, 3 months).
The advantage of ISO 8601 date/time formats is that the strings are (A) human readable, (B) they collate properly, (C) they're easy to parse and (D) its an international standard.
Some people "extend" ISO 8601's week notation, where a week of the year looks like 2011W32 (the 32nd week of 2011), to a quarter notation. Using this unofficial extension, the quarters of the year 2011 looks like:
2011Q1
2011Q2
2011Q3
2011Q4
How about using computed columns based on the payment date? I'd rather do this than have both a date and quarter/year that might get out of sync. On the other hand, I suppose it's possible that you may need the ability to have a different year/quarter than the date indicates in which case you'd need to keep them separate. I'd at least think about using computed columns though as that seems the best way to ensure integrity.
For something so simple, I would just keep 2 int columns, and to build up the (pivotal) dates using dateadd when required to use date ranges.
Another option is a single date column, for which you can store the first day in the quarter, so the 4 dates in a year would be 1-Jan, 1-Apr, 1-Jul, 1-Oct. You can extract the quarter, year easily using datepart Q and Y.
How about two ints, one for the year, and one for the quarter (1-4). Is that what you meant by option "a"?
Option "b" would work, but you have to remember to maintain the table every year or so.
I agree two ints are fine.
I would add an index consisting of both columns in case you need to sort or filter by year and quarter.
You could even use a single tinyint. It's enough for storing in the form YYQ,like 111, 112,113,114, 121...for a few years.
Storing quarter and year in database depends on how your payment data is being organized. Examples would be; how many different payment values are being inserted. Will the quarter/year ranges vary? etc.
One good technique for "defining" a quarter/year range is making a separate table with a "DateTime" field that identifies a quarter. You don't need to join the table, you just need to do programming in C# to figure out if the range falls within a particular pay quarter.
For example:
Table 1: Payments
-----------------
paymentID (int)
paymentAmount (double(7,2))
paymentDateTime (DateTime)
Table 2: QuarterYear
--------------------
quarterYearID (int)
dateFrom (date)
dateTo (date)
quarter (tinyint)
description (varchar)
Example Data
paymentID | paymentAmount | paymentDateTime
------------------------------------------------
1 | 20.24 | 2011-04-18 08:14:20
2 | 34.15 | 2011-04-19 07:42:15
3 | 51.87 | 2011-04-20 13:04:22
quarterYearID | dateFrom | dateTo | quarter | description
-----------------------------------------------------------------
1 | 2011-01-01 | 2011-03-31 | 1 | first quarter
2 | 2011-04-01 | 2011-06-30 | 2 | second quarter
3 | 2011-07-01 | 2011-09-31 | 3 | third quarter
4 | 2011-10-01 | 2011-12-31 | 4 | forth quarter
Example Query for getting all payments for "Quarter 2"
dateValue is a dynamically pulled variable from the payments table. C# will handle 'dateValue' value.
SELECT quarter FROM QuarterYear WHERE cast('dateValue' AS date) BETWEEN dateFrom AND dateTo;
I'm creating a DW that will contain data on financial securities such as bonds and loans. These securities are associated with payment schedules. For example, a bond could pay quarterly, while a mortgage would usually pay monthly (sometimes biweekly). The payment schedule is created when the security is traded and, in the majority of cases, will remain unchanged. However, the design would need to accommodate those cases where it does change.
I'm currently attempting to model this data and I'm having difficulty coming up with a workable design. One of the most commonly queried fields is "next payment date". Users often want to know when a security will pay next. Therefore, I want to make it as easy as possible for them to get the next payment date and amount for each security.
Also, users often run historical queries in which case they'd want the next payment date and amount as of a specific point in time. For example, they may want to look back at 1/31/09 and query the next payment dates (which would usually be in February 2009 for mortgages). It's also common that they want to query a security's entire payment schedule, which might consist of 360 records (30 year mortgage x 12 payments/year).
Since the next payment date and amount would be changing each month or even biweekly, these fields wouldn't seem to fit into a slow-changing dimension very well. It would probably make more sense to use a fact table, but I'm unsure of how to model it. Any ideas would be greatly appreciated.
Next payment date is an example of a "fact-free fact table". There's no measure, just FK's between at least two dimensions: the security and time.
You can denormalize the security to have a type-1 SCD (overwritten with each load) that has a few important "next payment dates".
I think it's probably better, however, to carry a few important payment dates along with the facts. If you have a "current balance" fact table for loans, then you have an applicable date for this balance, and you can carry previous and next payment dates along with the balance, also.
For the whole payment schedule, you have a special fact-free fact table that just has applicable date and the sequence of payment dates on into the future. That way, when the schedule changes, you can pick the payment sequence as of some particular date.
I would use a table (securityid,startdate, paymentevery, period) it could also include enddate, paymentpershare
period would be 1 for days, 2 for weeks, 3 for months, 4 for years.
So for security 1 that started paying weekly on 3/1/2009, then the date changed to every 20 days on 4/2, then weekly after 5/1/2009, then to monthly on 7/1/2009, it would contain:
1,'3/1/2009',1,2
1,'4/2/2009',20,1
1,'5/1/2009',1,2
1,'7/1/2009',1,3
To get the actual dates, I'd use an algorithm like this:
To know the payment dates on security 1 from 3/5/2009 to 5/17/2008:
Find first entry before 3/5 = 3/1
Loop:
Get next date that's after 3/5 and before the next entry (4/2 - weekly) = 3/8
Get next date that's before next the entry (4/2) = 3/15
Get next date that's before next the entry (4/2) = 3/22
Get next date that's before next the entry (4/2) = 3/29
Next date >4/2 switch to next entry:
Loop:
Get next date that's after 4/2 and before the next entry (5/1 - every 20 days) = 4/22
Next date 5/12 is AFTER next entry 5/1, switch to next entry
Loop:
Get next date that's after 5/1 and before the lastdate (5/17 - weekly) = 5/8
Get next date that's before the lastdate = 5/15
Next date > 5/17
The dates between 3/5/2009 and 5/17/2008 would be 3/8,3/15,3/22,3/29,4/22,5/8,5/15
Why not store the next payment date as the amount of days from the date of the current payment?
Further clarification:
There would be a fact for every past payment linked to some date dimension. Each one of these facts will have a field next payment in which will be an integer. The idea is that the date of the current payment + next payment in will be the date of the next payment fact. This should be able to cater for everything.