I've been working on some SQL code to measure efficiency in real-time for some production data. Here's a quick background:
Operators will enter in data for specific sub assemblies. This data looks something like this:
ID PO W/S Status Operator TotalTime Date
60129515_2000_6_S025 107294 S025 Completed A 38 05/08/2020
60129515_2000_7_S025 107294 S025 Completed A 46 05/08/2020
60129515_2000_8_S025 107294 S025 Completed A 55 05/08/2020
60129515_2025_6_S020 107295 S020 Completed B 58 05/08/2020
60129515_2025_7_S020 107295 S020 Completed B 47 05/08/2020
60129515_2025_8_S020 107295 S020 Completed B 45 05/08/2020
60129515_2000_1_S090 107294 S090 Completed C 33 05/08/2020
60129515_2000_2_S090 107294 S090 Completed C 34 05/08/2020
60129515_2000_3_S090 107294 S090 Completed C 21 05/08/2020
The relevant columns are the Operator, TotalTime and Date (note that the date is stored as varchar(50) because it plays nicer with Microsoft PowerApps that way).
What I need to do is:
Aggregate the sum of "TotalTime" grouped by Operator
Calculate the time elapsed based on a condition:
If between 7AM and 4PM, calculate the time elapsed since 7AM of the current day
If after 4PM, return the total time between 7AM and 4PM of the current day
Divide the SUM(TotalTime) by the TimeElapsed (AKA the first list item / second list item) in order to get a rough estimate of labor hours worked vs. hours passed in the day.
This calculation would change every time the query was ran. This will allow the Microsoft PowerApp that is pulling this query to refresh the efficiency measure in real time. I've taken a stab at it already - see below:
SELECT
md.Operator,
CASE
WHEN DATEADD(HOUR, -5, GETUTCDATE()) > CONVERT(DATETIME, CONVERT(DATE, DATEADD(HOUR, -5, GETUTCDATE()))) + '7:00' AND GETDATE() < CONVERT(DATETIME, CONVERT(DATE, DATEADD(HOUR, -5, GETUTCDATE()))) + '15:45'
THEN (SUM(isNull(md.TotalTime, 0)) + SUM(isNull(md.DelTime, 0))) * 1.0 / DATEDIFF(MINUTE, CONVERT(DATETIME, CONVERT(DATE, DATEADD(HOUR, -5, GETUTCDATE()))) + '7:00' , DATEADD(HOUR, -5, GETUTCDATE())) * 100.0
ELSE (SUM(isNull(md.TotalTime, 0)) + SUM(isNull(md.DelTime, 0))) / 420 * 100.0
END AS OpEfficiency
FROM
[Master Data] AS md
WHERE
md.[Date] = CONVERT(varchar(50), DATEADD(HOUR, -5, GETUTCDATE()), 101)
GROUP BY
md.Operator
Note: the DelTime is a different column regarding delay times. I am also converting back from UTC time to avoid any time zone issues when transferring to PowerApps.
However, this is horribly inefficient. I am assuming it is because the Date needs to be converted to datetime every single time. Would it work better if I had a calculated column that already had the date converted? Or is there a better way to calculate time elapsed since a certain time?
Thanks in advance.
There are a few things you can do to increase efficiency considerably. First, you want to make sure SQL can do a simple comparison when selecting rows, so you'll start by calculating a string to match your date on since your [Date] field is a string not a date.
Second, calculate the minutes in your shift (either 540 for a full shift or scaled down to 0 at 7 AM exactly) ahead of time so you aren't calculating minutes in each row.
Third, when summing for operators, use a simple sum on the minutes and calculate efficiency from that sum and your pre-calculated shift so far minutes.
One note - I'm casting the minutes-so-far as FLOAT in my example, maybe not the best type but it's clearer than other decimal types like DECIMAL(18,6) or whatever. Pick something that will show the scale you want.
My example uses a Common Table Expression to generate that date string and minutes-so-far FLOAT, that's nice because it fits in a direct query, view, function, or stored procedure, but you could DECLARE variables instead if you wanted to.
By filtering with an INNER JOIN on the [Date] string against the pre-calculated TargetDate string, I make sure the data set is pared down to the fewest records before doing any math on anything. You'll definitely want to INDEX [Date] to keep this fast as your table fills up.
All these together should give a pretty fast query, good luck
with cteNow as ( --Calculate once, up front - date as string, minutes elapsed as FLOAT (or any non-integer)
SELECT CASE WHEN 60*DATEPART(HOUR, GETUTCDATE())+DATEPART(MINUTE, GETUTCDATE()) > 60*21
--4PM in UTC-5, expressed in minutes
THEN CONVERT(float,(16-7)*60) --minutes in (4 PM-7 AM) * 60 minutes/hour
ELSE --Assume nobody is running this at 6 AM, so ELSE = between 7 and 4
CONVERT(float,60*DATEPART(HOUR, GETUTCDATE()) + DATEPART(MINUTE, GETUTCDATE()) - ((7+5)*60))
--Minutes since midnight minus minutes from midnight to 7 AM, shifted by
--UTS offset of 5 hours
END as MinutesToday --Minutes in today's shift so far
, FORMAT(DATEADD(HOUR,-5,GETUTCDATE()),'MM/dd/yyyy') as TargetDate --Date to search for
--as a string so no conversion in every row comparison. Also, index [Date] column
)
SELECT md.Operator, SUM(md.TotalTime) as TotalTime, SUM(md.TotalTime) / MinutesToday as Efficiency
FROM [Master Data] AS md INNER JOIN cteNow as N on N.TargetDate = md.[Date]
GROUP BY md.Operator, MinutesToday
BTW, you didn't make allowances for lunch or running before 7 AM, so I also ignored those. I think both could be addressed in cteNOW without adding much complexity.
Related
I have a SQL query like
SET THIS_YEAR_END = '2022-11-01';
SET THIS_YEAR_START = DATEADD(DAY, -4*7+1, $THIS_YEAR_END);
SET LAST_YEAR_END = '2021-11-02';
SET LAST_YEAR_START = DATEADD(DAY, -4*7+1, $LAST_YEAR_END);
select end_date from (
select * from data
where DATE>= DATEADD(DAY, -27 * 7, $LAST_YEAR_START))
AND END_DATE BETWEEN CUST.END_DATE - 26 * 7 AND CUST.END_DATE - 7
I'm confused with this dateadd function in SQL. Can anyone please explain what exactly it's doing?
Here is the docs: https://docs.snowflake.com/en/sql-reference/functions/dateadd.html
Basically DATEADD is adding a specified value to a certain date. The first parameter is indicating the units of time that you want to add (e.g. DAY or MONTH), the second parameter specifies the number of units (e.g. number of days or number of months) and the third paramter is the date, to which you want to add something.
In your example in line 2 you are reducing THIS_YEAR_END by -4*7+1 (=-27) days.
I get an time difference value of 5.5 which should subtract 5 hours and 30 minutes from current date time .
select Dateadd(HH, -5.5,GETUTCDATE()),GETUTCDATE()
In output it only subtracts 5 hours .
I always get an value in form of this offset 5.5 or 13.5 likewise depending on the timezone..
Is this possible ???
I always get an value in form of this offset 5.5 or 13.5 likewise depending on the timezone..
Then you can multiply the value by 60 and use minute datepart
select Dateadd(MINUTE, -5.5 * 60,GETUTCDATE())
time arithmetic always works in whole units - use -330 minutes instead
select Dateadd(MINUTE, -330,GETUTCDATE()),GETUTCDATE()
the input to the function is intended to be an integer (whole number) - you may be finding that -5.5 is truncating off the decimal part
Yes this can be done. First, you need to convert your decimal offset into minutes. Then you can use DATEADD as before.
DECLARE #Offset DECIMAL(18, 2) = 5.5;
DECLARE #Mins INT = 60 * #Offset;
-- Offsetting by decimal time values.
SELECT
DATEADD(MINUTE, #Mins, GETDATE())
;
I have a table with call logs. I need to assign time slots for next call based on which time slot the phone number was reachable in.
The relevant columns of the table are:
Phone Number | CallTimeStamp
CallTimeStamp is a datetime object.
I need to calculate the following:
Time Slot: From the TimeStamp, I need to calculate the count for each time slot (eg. 0800-1000, 1001-1200, etc.) for each phone number. Now, if the count is greater than 'n' for a particular time slot, then I need to assign that time slot to that number. Otherwise, I select a default time slot.
Weekday Slot: Same as above, but with weekdays.
Priority: Basically a count of how many times a number was reached
Here's I have gone about solving these issues:
Priority
To calculate the number of times a phone number is called is straight forward. If a number exists in the call log, I know that it was called. In that case, the following query will give me the call count for each number.
SELECT DISTINCT(PhoneNumber), COUNT(PhoneNumber) FROM tblCallLog
GROUP BY PhoneNumber
However, my problem is that I need to change the values in the field Count(PhoneNumber) based on the value in that column itself. How do I go about achieving this? (eg. If Count(PhoneNumber) gives me a value > 20, I need to change it to 5).
Time Slot / Weekday
This is where I'm completely stumped and am looking for the "database" way of doing things.
Unfortunately, I can't get out of my iterative process of thinking. For example, if I was aggregating for a certain phone number (say '123456') and in a certain time slot (say between 0800-1000 hrs), I can write a query like this:
DECLARE #T1Start time = '08:00:00.0000'
DECLARE #T2End time = '10:00:00.0000'
SELECT COUNT(CallTimeStamp) FROM tblCallLog
WHERE PhoneNumber = '123456' AND FORMAT(CallTimeStamp, 'hh:mm:ss') >= #T1Start AND FORMAT(CallTimeStamp, 'hh:mm:ss') < #T2End
Now, I could go through each and every Distinct Phone Number in the table, count the values for each time slot and then assign a slot value for the phone number. However, there has to be a way that does not involve me iterating through a database.
So, I am looking for suggestions on how to solve this.
Thanks
You can use DATEPART Function to get week day slot.
To calculate time slot you can try dividing number of minutes from beginning of day and dividing it by size of the time slot. It would return you slot number. You can use either CASE statement to translate it to proper string or look table where you can store slot descriptions.
SELECT
PhoneNumber
, DATEPART(WEEKDAY, l.CallTimeStamp) AS DayOfWeekSlot
, DATEDIFF(MINUTE, CONVERT(DATE, l.CallTimeStamp), l.CallTimeStamp) / 120 AS TwoHourSlot /*You can change number of minutes to get different slot size*/
, COUNT(*) AS Count
FROM tblCallLog l
GROUP BY PhoneNumber
, DATEPART(WEEKDAY, l.CallTimeStamp)
, DATEDIFF(MINUTE, CONVERT(DATE, l.CallTimeStamp), l.CallTimeStamp) / 120
You could try this to return the phone number, the day of the week and a 2 hour slot. If the volume of calls is greater than 20 the value is set to 5 (not sure why to 5?). The code for the 2 hour section is adapted from this question How to Round a Time in T-SQL where the value 2 in (24/2) is the number of hours in your time period.
SELECT
PhoneNumber
, DATENAME(weekday,CallTimeStamp) as [day]
, CONVERT(smalldatetime,ROUND(CAST(CallTimeStamp as float) * (24/2),0)/(24/2)) AS RoundedTime
, CASE WHEN COUNT(*) > 20 THEN 5 ELSE COUNT(*) END
FROM
tblCallLog
GROUP BY
PhoneNumber
, DATENAME(weekday,dateadd(s,start_ts,'01/01/1970'))
I only want to keep durations less than 10 minutes long. my current code is as follows:
Duration = DateDiff(ss, TimeBegin, TimeEnd)
TimeBegin and TimeEnd are in TIME format. Obviously Duration right now comes back as:
00:00:07
That's where I'm running into trouble. Can I use a statement that looks like this:
<= 00:10:00 or <= '00:10:00'
Essentially I want:
Duration = (Datediff(ss, TimeBegin, TimeEnd) *ONLY IF LESS THAN 10 MINUTESg)
I already state earlier in the query that if no result is returned to create a NULL, so when a duration is Greater than 10 minutes, I just want it to be ignored as if it didn't exist.
DateDiff(ss, TimeBegin, TimeEnd) gives you the difference in seconds. Just use a Case statement to return the value only if that's under 600 (...ELSE Null is implied):
set #Duration = CASE
WHEN DateDiff(ss, #TimeBegin, #TimeEnd) < 600
THEN DateDiff(ss, #TimeBegin, #TimeEnd)
END;
Agreed with #aucuparia. But (this is for topic starter) be careful with using datediffs is seconds instead of minutes. Their behavior isn't the same, like months/years datediffs. I mean server rounds your operands:
select datediff(mi,'2014-05-15 19:00:00','2014-05-15 19:10:59')
is not the same like
select datediff(ss,'2014-05-15 19:00:00','2014-05-15 19:10:59')
Just execute them both and see the difference. The first one still is 10-minutes difference, but the second will be cut off by 600-seconds 'where' clause.
I have a stored procedure that works correctly, but don't understand the theory behind why it works. I'm indentifying a consecutive period of time by utilizing a datepart and dense rank (found solution through help elsewhere).
select
c.bom
,h.x
,h.z
,datepart(year, c.bom) * 12 + datepart(month, c.bom) -- this is returning a integer value for the year and month, allowing us to increment the number by one for each month
- dense_rank() over ( partition by h.x order by datepart(year, c.bom) * 12 + datepart(month, c.bom)) as grp -- this row does a dense rank and subtracts out the integer date and rank so that consecutive months (ie consecutive integers) are grouped as the same integer
from
#c c
inner join test.vw_info_h h
on h.effective_date <= c.bom
and (h.expiration_date is null or h.expiration_date > c.bom)
I understand in theory what is happening with the grouping functionality.
How does multiplying year * 12 + month work? Why do we multiply the year? What is happening in the backend?
The year component of a date is an integer value. Since there are 12 months in a year, multiplying the year value by 12 provides the total number of months that have passed to get to the first of that year.
Here's an example. Take the date February 11, 2012 (20120211 in CCYYMMDD format)
2012 * 12 = 24144 months from the start of time itself.
24144 + 2 months (february) = 24146.
Multiplying the year value by the number of months in a year allows you to establish month-related offsets without having to do any coding to handle the edge cases between the end of one year and the start of another. For example:
11/2011 -> 24143
12/2011 -> 24144
01/2012 -> 24145
02/2012 -> 24146