Days count across the months with given start date and end date - snowflake-cloud-data-platform

I have a snippet of a table as below :
Can somebody help me with the query that gives me report as below:

If you don't want to use a calendar dimension as Mike suggested (which is a good idea) the best option may be a user defined table function (UDTF). This UDTF does not depend on the DAYS_COUNT column. It calculates the days in each month and only requires a start and end date. It will generate as many rows as are required to fill the months in between.
select * from a, table(DAYS_IN_MONTHS(START_DATE, END_DATE));
create or replace function DAYS_IN_MONTHS(START_DATE date, END_DATE date)
returns table(MONTHS string, DAYS float)
language javascript
strict immutable
as
$$
{
initialize: function (argumentInfo, context) {},
processRow: function (row, rowWriter, context) {
let year = row.START_DATE.getFullYear();
let month = row.START_DATE.getMonth();
let endYear = row.END_DATE.getFullYear();
let endMonth = row.END_DATE.getMonth();
let isDone = year > endYear || (year == endYear && month > endMonth);
if (year == endYear && month == endMonth) {
rowWriter.writeRow({MONTHS: `${year}-${(""+(month+1)).padStart(2, '0')}`,DAYS: row.END_DATE.getDate() - row.START_DATE.getDate()});
isDone = true;
}
d = row.START_DATE
while (!isDone) {
if (year == endYear && month == endMonth) {
rowWriter.writeRow({MONTHS: `${year}-${(""+(month+1)).padStart(2, '0')}`,DAYS: row.END_DATE.getDate() - (d.getDate() - 1) });
isDone = true;
} else {
rowWriter.writeRow({MONTHS: `${year}-${(""+(month+1)).padStart(2, '0')}`,DAYS: new Date(year, month + 1, 0).getDate() - (d.getDate() - 1) });
month++;
if (month == 12) {
month = 0;
year++;
}
}
d = new Date(year, month, 1);
}
},
finalize: function (rowWriter, context) {},
}
$$;

It would be very helpful if you'd have a calendar dimension for something like this. As an example, you can create one with something like this:
CREATE TEMP TABLE calendar AS
SELECT dateadd(day,seq4(),'2017-01-01'::date) as cal_date,
date_trunc(month,cal_date) as cal_month_start,
dateadd(day,-1,(dateadd(month,1, cal_month_start))) as cal_month_end
FROM table(generator(rowcount => 10000));
With this calendar table, you can then join to it using the start and end dates and aggregate on that date to get the results. Using a CTE to replicate your data:
WITH x AS (
SELECT member_id, start_date, end_date
FROM (VALUES ('461043068_02','2018-08-07'::date,'2018-08-17'::date),
('461043068_01','2019-05-28'::date,'2019-06-28'::date)
) y (member_id, start_date, end_date)
)
Now, you can query x and join to calendar as such:
SELECT member_id, cal_month_start, count(*)
FROM x
JOIN calendar c
ON c.cal_date between dateadd(day,1,x.start_date) and x.end_date
GROUP BY 1,2;
This gives you the results you are looking for. Please note the need to add 1 to the start_date, so that you don't count the "edges" of your date ranges twice.
Also, I didn't format the cal_month_start in my query, but you can do that using a TO_VARCHAR() function, if needed.

Related

How to call mysql function in snowflake UDF

I am trying to write below UDF in snowflake.But it doesnt allow to use sql functions like TO_DATE,DATE_ADD,dayofweek . Any alternative idea would be helpful. Thanks.
CREATE OR REPLACE FUNCTION getShippingDate(deliveryDate varchar,deliveryCountryCode varchar, holidayList varchar,deliveryDays varchar)
RETURNS VARCHAR
LANGUAGE JAVASCRIPT
AS $$
deliveryPeriod = 0
weekDay = 0
deliveryDate= TO_DATE(deliveryDate)
if(deliveryCountryCode != 'IN') {
deliveryPeriod = 2
}
else {
deliveryPeriod = deliveryDays + 1
}
if(deliveryPeriod <= 0) {
deliveryPeriod = 1
}
/* substract delivery period from delivery date */
deliveryDate = DATEADD(Day ,-deliveryPeriod, to_date(deliveryDate))
weekDay = dayofweek(deliveryDate)
/* if shipping date falls on sunday then substract 2 days */
if (weekDay == 0) {
deliveryDate = DATEADD(Day ,-2, to_date(deliveryDate))
}
/* if shipping date falls on saturday then substract 1 days */
if(weekDay == 6){
deliveryDate = DATEADD(Day ,-1, to_date(deliveryDate))
}
/* check if shipping date falls on holiday then substract 1 */
if(charindex(deliveryDate , holidayList) > 0) {
deliveryDate = DATEADD(Day ,-1, to_date(deliveryDate))
}
return deliveryDate
$$```
It is relatively easy to create date functions in JavaScript. Look at example below.
Just remember:
Please use the DATE SQL datatype for input and output
Parameters must be "Quoted" if you want to reference them in MixedCase
Use a semicolon ; to terminate each statement
CREATE OR REPLACE FUNCTION getShippingDate("deliveryDate" DATE, "offset" FLOAT)
RETURNS DATE
LANGUAGE JAVASCRIPT
AS $$
function day_add(dt, days) {
return new Date(dt.getFullYear(), dt.getMonth(), dt.getDate() + offset);
}
return day_add(deliveryDate, offset);
$$;
SELECT getShippingDate(CURRENT_DATE, -2);
new Date() does some magic when you add or subtract outside the days of the month.
Implementing your function as a scalar SQL UDF would solve the problem in this case. Or you could implement your own javascript date math. For example. The good news is that you can call JS UDFs from your SQL UDFs.
You may need to try like below:
CREATE OR REPLACE PROCEDURE dbo.usp_test()
returns VARCHAR
LANGUAGE javascript
EXECUTE AS CALLER
AS
$$
snowflake.createStatement( {sqlText: "SET dt = DATEADD(MONTH, 1,CURRENT_TIMESTAMP)"} ).execute();
return 'success';
$$
;
-- CALL ds_work.ddd.usp_test()

Strange results using dates in linq queries using EF Core

I'm getting strange results using trying to do a simple query against a date column using Linq and EF Core.
If I run the query using a date from a list of DateTime I get no results. If I substitute DateTime.Now and add a negative number of days so that if matches the date in the list of DateTimes then the query returns results as expected.
So what is the difference between DateTime.Now and another DateTime object?
In practice, why would this work (rewinding now by 30 days in the first example gives the same date as datesToCheck[0] in the second):
var reports = from r
in db.DailyReports
.Where(r => r.UserId.Equals(currentuser.Identity.Name)
&& r.Date > DateTime.Now.AddDays(-30))
select r;
But not this:
var reports = from r
in db.DailyReports
.Where(r => r.UserId.Equals(currentuser.Identity.Name)
&& r.Date > datesToCheck[0])
select r;
The database is SQL Server 2017, the column is a non-nullable smalldatetime
The datesToCheck list is generated thus:
var datesToCheck = new List<DateTime>();
var startDate = DateTime.Now;
//get Monday date three weeks ago
if (DateTime.Now.DayOfWeek != DayOfWeek.Monday)
{
while (startDate.DayOfWeek != DayOfWeek.Monday)
{
startDate = startDate.AddDays(-1);
}
}
startDate = startDate.AddDays(-21);
while (startDate < DateTime.Now)
{
if (startDate.DayOfWeek != DayOfWeek.Saturday || startDate.DayOfWeek != DayOfWeek.Sunday)
{
datesToCheck.Add(startDate);
startDate = startDate.AddDays(1);
}
}
The same behavior exists in EF6 and, as far as I know, all versions of EF. Basically, the compiler isn't clever enough to decide if datesToCheck[0] should be evaluated or converted to SQL. The query will work if you store the value in a variable and then use the variable in the LINQ query. See also: Why can't we use arrays in Entity Framework queries?
You probably have some datatype issue, Try:
DateTime datesToCheck = DateTime.Now.AddDays(-30);
var reports = from r
in db.DailyReports
.Where(r => r.UserId.Equals(currentuser.Identity.Name)
&& r.Date > datesToCheck )
select r;

calculate date difference between two dates with angular js

I want to calculate date difference between two dates with angular js. I used this code:
$scope.formatString = function(format) {
var year = parseInt(format.substring(0,5));
var month = parseInt(format.substring(6,8));
var day = parseInt(format.substring(9,10));
var date = new Date(year, month-1, day);
return date;
}
$scope.dayDiff = function(fromdate,todate){
var date2 = new Date($scope.formatString(fromdate));
var date1 = new Date($scope.formatString(todate));
var timeDiff = Math.abs(date2.getTime() - date1.getTime());
var diffDays = Math.ceil(timeDiff / (1000 * 3600 * 24));
return diffDays;
}
But it is right for within same month but it can't do right calculation between two dates in different months.
For example date1 = 2016.2.12, date2 = 2016.2.18 (Ok)
For example date1 = 2016.2.12, date2 = 2016.3.1 (It is not right)
Why not try using the String split function (docs) to parse your date, split will produce an array of the year, month, and day in that order based on the format you're inputting. From there you can reference the array pieces by index, a bit easier than trying to get your substring offsets lined up correctly:
$scope.formatString = function(format) {
var pieces = format.split('.'),
year = parseInt(pieces[0]),
month = parseInt(pieces[1]),
day = parseInt(pieces[2]),
date = new Date(year, month - 1, day);
return date;
}
Take a look at this working example: http://plnkr.co/edit/GgXjqA76IX5bzQEP56ux?p=preview
It is better to use a ready-made library. Because you do not know that in the future you will still need to work with dates. I recommend sugarjs.
The difference between the dates can be found in the following manner.
Date.range('2012.01.01', '2013.01.01').every('days').length

Converting day, weeknr and year to date

I've been given an excel document in which worktime information is noted, this document contains certain columns which are being read by using SSIS in visual studio, after that the information is writen to a Database.
The week and year column contain the week number and the year, the columns Monday up to Friday contain information about how many working hours have been spent on a certain task on that day of the week.
What I'd like to do is take the WeekNr, Year and Day and convert these into a date. I've been trying to accomplish this by using a script component that converts a day number, week number and year to a date but so far I haven't been able to get the day number from the columns. In my opinion it would work best if used with a start and end date taking the first and last date of that week.
So my question is if someone knows how to accomplish this, or if I should try a different approach.
The script component:
public override void Input0_ProcessInputRow(Input0Buffer Row, CultureInfo cultureInfo, int day )
{
DateTime firstDayOfYear = new DateTime(Int32.Parse(Row.Jaar), 1, 1);
int firstWeek = cultureInfo.Calendar.GetWeekOfYear(firstDayOfYear, cultureInfo.DateTimeFormat.CalendarWeekRule, cultureInfo.DateTimeFormat.FirstDayOfWeek);
int dayOffSet = day - (int)cultureInfo.DateTimeFormat.FirstDayOfWeek + 1;
Row.TaskDates = firstDayOfYear.AddDays((Int32.Parse(Row.Week) - (firstWeek + 1)) * 7 + dayOffSet + 1);
}
Based on this answer, I think you want something like the following. This result gives you Monday's date, so you can just AddDays based on the column day of the week.
DateTime jan1 = new DateTime(Int32.Parse(Row.Jaar), 1, 1);
int daysOffset = DayOfWeek.Monday - jan1.DayOfWeek;
DateTime firstMonday = jan1.AddDays(daysOffset);
var cal = CultureInfo.CurrentCulture.Calendar;
int firstWeek = cal.GetWeekOfYear(firstMonday, CalendarWeekRule.FirstFullWeek, DayOfWeek.Monday);
var weekNum = Int32.Parse(Row.Week);
if (firstWeek <= 1)
{
weekNum -= 1;
}
var mondaysDate = firstMonday.AddDays(weekNum * 7);
var tuesdaysDate = mondaysDate.AddDays(1);

ndb query error with datetime field - Google App Engine

I'm having a problem and I don't find any information about.
I define a field in my model like this.
class Dates(ndb.model):
...
date = ndb.DateTimeProperty(required = True) # I want to store date and time
...
Later I try a query (now I want all the dates for a day, I don'tn mind the time):
kl = Dates.query(ndb.AND(Dates.date.year == year,
Dates.date.month == month,
Dates.date.day == day),
ancestor = customer.key).fetch(keys_only = True)
dates = ndb.get_multi(kl)
But I get this error log:
AttributeError: 'DateTimeProperty' object has no attribute 'year'
I don't know why. I've tried Dates.date() == date, Dates.date == date (<-DateTime obj), ...
My DB is still empty but I suppose this doesn't mind because I'll never have dates for every possible days.
Anybody knows why? Should I go with GQL instead?
You can use "range" queries for this. See example below.
import datetime
date = datetime.datetime.strptime('02/19/2013', '%m/%d/%Y')
kl = Dates.query(
ndb.AND(Dates.date >= date),
Dates.date < date + datetime.timedelta(days=1))
Will fetch all datetime's with 02/19/2013.
What you are trying to achieve is not really possible, because you can only query for the whole date and not for some parts of it.
In order to achieve what you are trying there I would suggest you to add few more properties to your model:
class Dates(ndb.model):
...
date = ndb.DateTimeProperty(requiered=True)
date_year = ndb.IntegerProperty()
date_month = ndb.IntegerProperty()
date_day = ndb.IntegerProperty()
...
You could update these values on every save or you could use Model Hooks to do it automagically and then your new query will become:
kl = Dates.query(ndb.AND(Dates.date_year == year,
Dates.date_month == month,
Dates.date_day == day),
ancestor=customer.key).fetch(keys_only=True)
dates = ndb.get_multi(kl)
Use a DateProperty. Then you can use a simple == query:
>>> import datetime
>>> from google.appengine.ext.ndb import *
>>> class D(Model):
... d = DateProperty()
...
>>> d = D(d=datetime.date.today())
>>> d.put()
Key('D', 9)
>>> d
D(key=Key('D', 9), d=datetime.date(2013, 2, 20))
>>> D.query(D.d == datetime.date.today()).fetch()
[D(key=Key('D', 9), d=datetime.date(2013, 2, 20))]
I expanded #Guido van Rossum code snippet to include <> and timedelta for calculations, mostly for my own satisfaction
import datetime
from datetime import timedelta
from google.appengine.ext.ndb import *
class D(Model):
d = DateProperty()
now = datetime.date.today()
date1 = now-timedelta(+500)
date2 = now-timedelta(+5)
d1 = D(d=now)
d2 = D(d=date1)
d3 = D(d=date2)
d1.put()
d2.put()
d3.put()
date2 = now-timedelta(+50)
result1 = D.query(D.d == now).fetch(4)
result2 = D.query(D.d > date2).fetch(2)
result3 = D.query(D.d < date2).fetch(2)
result4 = D.query(D.d >= date2, D.d <= now).fetch(2)
print result1
print "+++++++"
print result2
print "+++++++"
print result3
print "+++++++"
print result4

Resources