Requesting some inputs on testing the feature of snowflake as per the documentation:
"To honor the data retention period for these child objects (schemas or tables), drop them explicitly before you drop the database or schema."
URL --> https://docs.snowflake.com/en/user-guide/data-time-travel.html#dropped-containers-and-object-retention-inheritance
Example:
Use case --> Schema & table time travel feature testing for inheritance. Schema--> "S", Table --> "T"
As per definition --> If table "T" has a time travel of 7 days under the schema "S" which has its value as 5 days, then 5 days of time travel would be considered.
What is to be tested --> I want to retain the time travel for the same table to be at "7 days".
Regards,
Somen Swain
Related
I have two tables case and party. Both of them are from BigQuery. In the case table, there is a date field in the DATETIME type in BigQuery. I want to plot a chart to show the number of cases per month. It works perfectly when I only use the case table.
However, when I blend the case table with the party table, the aggregation doesn't apply on the date field anymore.
This is how I join the tables.
How do make the aggregation of date & time fields in blends work as normal?
Update
I have tried to create a custom field with TODATE(date, 'RFC_3339', '%Y-%m-%d'), but the aggregation is still not applied to the custom field.
minimal example
Create a.csv in local.
a_id,date
1,2022-05-25T00:00:00
2,2022-05-25T00:00:00
3,2022-06-01T00:00:00
4,2022-06-02T00:00:00
Create b.csv in local.
a_id,b_id
1,1
1,2
2,3
3,4
4,5
4,6
Upload a.csv and b.csv into a Data Studio report.
Blend a.csv and b.csv with inner joining on a_id field. I removed "Record Count" from metrics in both tables and added all fields as dimensions.
Add a "Table" type chart using the blend created in the previous step.
Put date field as the only one dimension and select "Year Month" as the type.
Put a_id field as the only one metrics and select "Count Distinct" as aggregation.
actual result
date (Year Month)
a_id
May 2022
2
Jun 2022
1
Jun 2022
1
expected result
date (Year Month)
a_id
May 2022
2
Jun 2022
2
example report
Here
This is a bug, and it was already reported in Google's DataStudio IssueTracker.
https://issuetracker.google.com/issues/179280180
You probably want to star the issue (Google prioritize issues by the number of affected users).
Also, in the same thread, the reporting user suggests to use the above formula as a workaround.
YearMonthAsDate = DATE(YEAR(date_tran),MONTH(date_tran),1)
It worth a try.
You are changing the Date dimension in the table but in order to aggregate it on a monthly basis but joins the rows on a date base.
The solution is to change the Dimension to Month year inside the blend settings, you can do this just the same way you change a field type inside a table.
I have data generated on daily basis.
let me explain through a example:
On World Market, the price of Gold change on seconds interval basis. and i want to store that price in Redis DBMS.
Gold 22 JAN 11.02PM X-price
22 JAN 11.03PM Y-Price
...
24 DEC 11.04PM X1-Price
Silver 22 JAN 11.02PM M-Price
22 JAN 11.03PM N-Price
I want to store this data on daily basis. want to apply ML (Machine Leaning) on last 52 Week data. Is this possible?
Because As my knowledge goes. redis work on Key Value.
if this is possible. Can i get data from a specific date(04 July) and DateRange(01 Feb to 31 Mar)
In redis, a Sorted Set is appropriate for time series data. If you score each entry with the timestamp of the price quote you can quickly access a whole day or group of days using the ZRANGEBYSCORE command (or ZSCAN if you need paging).
The quote data can be stored right in the sorted set. If you do this make sure each entry is unique. Adding a record to a sorted set that is identical to an existing one just updates the existing record's score (timestamp). This moves the old record to the present and erases it from the past, which is not what you want.
I would recommend only storing a unique key/ID for each quote in the sorted set, and store the data in it's own key or hash field. This will allow you to create additional indexes for the data as needed and access specific records more easily if necessary.
Salesforce:
Every account has an agreement with my company. Once they have an agreement in place, they can start ordering products.
In the account record, I have a date field called "Agreement Approved On".
Ordered products are represented as "assets".
I want to create another field in the account record called "First Order Amount". This field should be populated dynamically, and should calculate the following:
Look for all of the account's assets created after the date field "Agreement Approved On".
Summarize the total of all assets ordered during the first order (within the same day).
Within the same day:
Let's assume the agreement was approved on January 1 2017. The
account started ordering on January 5 2017.
During January 5 2017, the account made 5 orders (8 different
assets), total is $1,000. This is the first order total - within the same
day.
After January 5 2017, the account made more orders, but they don't
count towards "first order amount".
So my question is - how do I create this "First Order Amount" field?
Thanks!
You need to setup a oncreate trigger on Asset taht will update the first order total amount field on Account.
In the trigger check whether now() > agreement approve date. If yes, check if there are any other assets with dateCreated < TODAY.
These two conditions will check whether there are any existing account assets and if the date is > agreement date. If there are existing assets, it means this is not the first order.
You can then get all asset amounts and add them and write the final amount to the account field.
See https://developer.salesforce.com/docs/atlas.en-us.apexcode.meta/apexcode/apex_triggers_context_variables.htm
for trigger syntax
and https://developer.salesforce.com/docs/atlas.en-us.soql_sosl.meta/soql_sosl/sforce_api_calls_soql_select_dateformats.htm for date functions and literals.
I want to load one table for data for say 1 month starting from 1 April to 30 April in successive manner.
i.e after loading data for 1 April, date should automatically increment to 2, load the data and increment and so on, till its 30 April.
Also, data of 2 April depends on 1 April data. So i cannot give a date range to load randomly.
How can I do it?
It would be preferable to get the loads done in single session run, instead of running the session for several times.
Sort the source data by date and use a Transaction Control transformation to enforce a commit every time the date changes.
I'm creating a DW that will contain data on financial securities such as bonds and loans. These securities are associated with payment schedules. For example, a bond could pay quarterly, while a mortgage would usually pay monthly (sometimes biweekly). The payment schedule is created when the security is traded and, in the majority of cases, will remain unchanged. However, the design would need to accommodate those cases where it does change.
I'm currently attempting to model this data and I'm having difficulty coming up with a workable design. One of the most commonly queried fields is "next payment date". Users often want to know when a security will pay next. Therefore, I want to make it as easy as possible for them to get the next payment date and amount for each security.
Also, users often run historical queries in which case they'd want the next payment date and amount as of a specific point in time. For example, they may want to look back at 1/31/09 and query the next payment dates (which would usually be in February 2009 for mortgages). It's also common that they want to query a security's entire payment schedule, which might consist of 360 records (30 year mortgage x 12 payments/year).
Since the next payment date and amount would be changing each month or even biweekly, these fields wouldn't seem to fit into a slow-changing dimension very well. It would probably make more sense to use a fact table, but I'm unsure of how to model it. Any ideas would be greatly appreciated.
Next payment date is an example of a "fact-free fact table". There's no measure, just FK's between at least two dimensions: the security and time.
You can denormalize the security to have a type-1 SCD (overwritten with each load) that has a few important "next payment dates".
I think it's probably better, however, to carry a few important payment dates along with the facts. If you have a "current balance" fact table for loans, then you have an applicable date for this balance, and you can carry previous and next payment dates along with the balance, also.
For the whole payment schedule, you have a special fact-free fact table that just has applicable date and the sequence of payment dates on into the future. That way, when the schedule changes, you can pick the payment sequence as of some particular date.
I would use a table (securityid,startdate, paymentevery, period) it could also include enddate, paymentpershare
period would be 1 for days, 2 for weeks, 3 for months, 4 for years.
So for security 1 that started paying weekly on 3/1/2009, then the date changed to every 20 days on 4/2, then weekly after 5/1/2009, then to monthly on 7/1/2009, it would contain:
1,'3/1/2009',1,2
1,'4/2/2009',20,1
1,'5/1/2009',1,2
1,'7/1/2009',1,3
To get the actual dates, I'd use an algorithm like this:
To know the payment dates on security 1 from 3/5/2009 to 5/17/2008:
Find first entry before 3/5 = 3/1
Loop:
Get next date that's after 3/5 and before the next entry (4/2 - weekly) = 3/8
Get next date that's before next the entry (4/2) = 3/15
Get next date that's before next the entry (4/2) = 3/22
Get next date that's before next the entry (4/2) = 3/29
Next date >4/2 switch to next entry:
Loop:
Get next date that's after 4/2 and before the next entry (5/1 - every 20 days) = 4/22
Next date 5/12 is AFTER next entry 5/1, switch to next entry
Loop:
Get next date that's after 5/1 and before the lastdate (5/17 - weekly) = 5/8
Get next date that's before the lastdate = 5/15
Next date > 5/17
The dates between 3/5/2009 and 5/17/2008 would be 3/8,3/15,3/22,3/29,4/22,5/8,5/15
Why not store the next payment date as the amount of days from the date of the current payment?
Further clarification:
There would be a fact for every past payment linked to some date dimension. Each one of these facts will have a field next payment in which will be an integer. The idea is that the date of the current payment + next payment in will be the date of the next payment fact. This should be able to cater for everything.