We transfer data from Oracle SQL to SQL Server in the near real-time data warehouse. (Oracle is a transactional database) The time changes twice a year for the data we use. We have been having issues when calculating DateTime differences and I have to work on a short-term and long-term solutions where I need advice.
New data coming in :
I can create tables from scratch and use datetimeoffset data field for all the time variables. Would this
display dates in SQL Server queries in the local time zone and save them in their actual UTC time? And when I
calculate the differences, would it give the correct difference?
Existing data without offset information:
I already have data converted into the current timezone. We have 2 timezones AEST and AEDT.
When we calculate time differences for KPIs it gives incorrect times. Any ideas on how it can be resolved?
Can creating a function that converts the dates and times into UTC and subtracts the difference and returns the value would work?
An example below :
DECLARE #startdate DateTime = '2022-10-01 23:13:00.000'; --UTC 2022-10-01 13:13:00.00
DECLARE #enddate DateTime = '2022-10-02 12:08:00.000' --UTC 2022-10-02 01:08:00.00
select CAST((#enddate - #startdate) as time(0)) 'Difference'
Time difference: 12:55:00
Actual Time Difference: 11:55:00
For your example, if you include the time zone info with the data, the date calculations work out
DECLARE #startdate datetimeoffset(0) = '2022-10-01 23:13:00 +10:00'; --UTC 2022-10-01 13:13:00.00
DECLARE #enddate datetimeoffset(0) = '2022-10-02 12:08:00 +11:00' --UTC 2022-10-02 01:08:00.00
declare #minutes int = datediff(minute, #startdate, #enddate);
select #startdate at time zone 'utc',
#enddate at time zone 'utc';
select [hours] = #minutes / 60,
[minutes] = #minutes - 60 * (#minutes / 60)
To address the existing data, you can do a one-time conversion to datetimeoffset by running it through at time zone with the appropriate time zone name. In my example above, I used UTC but you can find all of the supported time zones in sys.time_zone_info.
As to your first question about the internal storage format and display, I'm not sure if it uses UTC internally or not. As to display, it will, by default, output them with the time zone that you stored them with. That is, if you stored a given value with +10:00 as the offset, that's how it will display. You can convert via methods already described above or with switchoffset(). Lastly, if it matters, you can get just the offset with datepart noting that it returns the offset in number of minutes (to conform with datepart needing to return an int).
Related
I have a Stored Procedure that will run within a time period (between 2-3 am), but not always at the exact same time during that period.
As part of this procedure I need to do 3 steps:
1: Get the start of yesterday's date
So if today is 13/08/2020 13:51:02 I need the query to return 12/08/2020 00:00:00 and do this dynamically
2: Get the end of yesterday's date
In the above this would return 12/08/2020 23:59:59
3: Convert both values into Unix EPOCH timestamps
I have used in the past on a similar issue (that was less time-sensitive) the below bit of Code:
declare #yesterday date
set #yesterday = (SELECT convert(datetime, DATEADD(day, -1 ,getdate()), 23 ))
The Problem here is that this gives a value that is exactly 24 hours in the past, so if the SP is run at 2:15 am - it's time stamp will be different when it's run at 2:23 am or 2:53 am.
Once I've got a method of getting the start and end date to always be correct - I'll then use something like this solution to convert the Datetime into Epoch timestamps, unless someone who answers this question has a snazzy method of doing it all in one (for which I would be eternally grateful)
To get midnight yesterday, do this:
DECLARE #yesterday DATETIME
= DATEADD(DAY, -1, CAST(GETDATE() AS DATE));
To get midnight today do
DECLARE #today DATETIME = CAST(GETDATE() AS DATE);
To filter for times that happened yesterday, do
WHERE ts >= #yesterday AND ts < #today ...
The CAST operation truncates the date/time value returned by GETDATE() to midnight.
The form of WHERE with >= and < copes with the edge case correctly. It also happens to exploit an index on the column I called ts.
I have a question about converting UTC time zone to EST time zone since I am still new to SQL language and Azure platform. In log file, I used system built-in function "GetDate()" in log file to get the Date/Time. However, while accessing Azure database on my SSMS, using system-built in function (GetDate()) gives me the datetime in UTC time zone, which is 4 hours ahead of Eastern Time Zone (EST). I have asked similar question before for converting UTC to EST here
, and #DanGuzman helped me fix my code. But this question is more about converting UTC to EST (considering Daylight saving time dynamically). Below is my code so far, and I used this link as a reference. However, I would like to make my code dynamic so that I can keep using it for 2020, 2021 as well.
Below code works ONLY for 2019 (since Daylight Saving starts on March 10,2019 until November 3, 2019. Within the date range, below code forward one hour of EST time during daylight saving time range.
CREATE FUNCTION [dbo].[EST_GetDateTime]
(
-- no parameter
)
RETURNS datetime
AS
BEGIN
DECLARE
#EST datetime,
#StandardOffset int,
#DST datetime, -- Daylight Saving Time
#SSM datetime, -- Second Sunday in March
#FSN datetime -- First Sunday in November
-- get DST Range
set #EST = CAST(DATEADD(hh,-5,GETDATE()) AS DATETIME)
set #StandardOffset = 0
set #SSM = datename(year,#EST) + '0310' -- Set which day daylight saving start (for 2019, March 10)
set #SSM = dateadd(hour,2,dateadd(day,datepart(dw,#SSM)*-1+1,#SSM))
set #FSN = datename(year,#EST) + '1103' -- Set which day daylight saving start (for 2019, March 10)
set #FSN = dateadd(second,-1,dateadd(hour,2,dateadd(day,datepart(dw,#FSN)*-1+1,#FSN)))
-- add an hour to #StandardOffset if #EST is in DST range
if #EST between #SSM and #FSN
set #StandardOffset = #StandardOffset + 1
-- convert to DST
set #EST = CAST(DATEADD(hh,-5+#StandardOffset ,GETDATE()) AS DATETIME)
RETURN #EST
END
GO
Can someone please give any suggestions how to improve my existing code (able to work dynamically) so that I do not need to change the function every single year for adjusting day light saving.
Since Azure SQL is ahead of the on-premises version, I think you can use the syntax select getutcdate() at time zone 'UTC' at time zone 'Eastern Standard Time'. Also, I'd switch to getutcdate() as that should be invariant regardless of the server's TZ. More information on at time zone here.
One method is a function that converts SYSDATETIMEOFFSET() to EST and converts the result to datetime:
CREATE FUNCTION dbo.EST_GetDateTime()
RETURNS datetime
AS
BEGIN
RETURN(SELECT CAST(SYSDATETIMEOFFSET() AT TIME ZONE 'Eastern Standard Time' AS datetime));
END
GO
--example usage
SELECT dbo.EST_GetDateTime();
GO
This should do exactly what you need.
SELECT CONVERT(DATETIME,GETDATE() AT TIME ZONE (SELECT CURRENT_TIMEZONE_ID()) AT TIME ZONE 'Eastern Standard Time')
Explanation: This gets the current datetime in the server's local timezone using CURRENT_TIMEZONE_ID(). We then use AT TIME ZONE to make it a datetimeoffset, then we cut that datetimeoffset over to requested timezone... here 'Eastern Standard Time'. Lastly the whole thing is wrapped in a CONVERT() to cut the datetimeoffset objects over to a proper datetime datatype.
I have a process that is going to use table partitioning (each partition is 1 hour) and I need to handle the daylight savings flips when archiving out the data.
For instance, this past weekend it went from 1:59:59 to 1:00:00, so the second time that the partitioning code ran at 1:05am nothing would occur - the midnight hour had already been SWITCHed out.
However, when spring rolls around, the time goes from 1:59am to 3:00am, so when the job runs at 3:05, it would SWITCH out 2am.... leaving the data from the 1am hour in the original table.
In theory I can just look for the oldest not-current partition with data and flip that one (partition key is a default getdate() constraint), but I was wondering if there was some way to use AT TIME ZONE to determine that daylight savings had "occurred", so that we could have different code to handle that older hour still being out there.
Thanks.
Maybe something like this. Basically take your current getdate() and use AT TIME ZONE with any time zone that observes. Then use DATEPART tz and compare your current and before.
Regardless of what the server time zone is, using AT TIME ZONE will get you the offset for a particular datetime value in that time zone.
For comparison of before I think you'd need to use 2 hours, 1 for the switch and 1 for how far back you want to check.
Give this a look:
DECLARE #BeforeDate DATETIME = '2018-11-04 1:59' --Before the change
DECLARE #AfterDate DATETIME = '2018-11-04 3:00' --After the change
--use can use AT TIME ZONE with DATEPART tz which tells you offset in minutes
--I'm in central, any should work for any that observe the time change
--Your offset is different because of the change.
SELECT DATEPART(tz, #BeforeDate AT TIME ZONE 'Central Standard Time')
SELECT DATEPART(tz, #AfterDate AT TIME ZONE 'Central Standard Time')
--using the above you could possibly compare current offset to 2 hours prior to see if they changed. 2 hours, 1 for the switch and 1 for how far back you want to compare.
DECLARE #CurrentDate DATETIME = '2018-11-04 3:00' --"simulate" getdate() as of the time change
DECLARE #PriorOffSet INT = (SELECT DATEPART(tz, DATEADD(HOUR, -2, #CurrentDate) AT TIME ZONE 'Central Standard Time')) --You'd have to subtract 2 hours to account for the hour shift and the hour back you want to check.
DECLARE #CurrentOffset INT = (SELECT DATEPART(tz, #CurrentDate AT TIME ZONE 'Central Standard Time'))
SELECT #PriorOffSet, #CurrentOffset
IF #PriorOffSet <> #CurrentOffset
SELECT 'Time changed in the last hour'
ELSE
SELECT 'No time change in the last hour'
I have to query an SQL Server database and the table's values use Epoch time (an int. Here's an example - 1438005018). I am wondering how I can write a query so that I can say the following...
select
*
from
tablename
where
epochdate between 'yesterday at 12:00' and 'today at 12:00' --this is the part I'm not sure about.
Ideally, if it's easy, I'd like the query to use non-epoch logic as Epoch time confuses the crap out of me. Maybe there's a quick way of converting in SQL Server?
I posted a link above in the comments that may be a more practical solution if you're able to deploy functions in the database you're working with, but if you're only able to query, this is an option to try as well (this assumes SQL Server 2008 and above):
declare #todayepoch bigint, #yesterdayepoch bigint;
select #todayepoch =
cast((cast(dateadd(hour, 12,
cast(cast(sysutcdatetime() as date) as datetime)) as decimal(24,10))
- cast(cast('1970-01-01' as datetime) as decimal(24,10)))
*60.0*60.0*24.0 as int), -- + 18000, --Eastern time
#yesterdayepoch =
cast((cast(dateadd(hour, -12,
cast(cast(sysutcdatetime() as date) as datetime)) as decimal(24,10))
- cast(cast('1970-01-01' as datetime) as decimal(24,10)))
*60.0*60.0*24.0 as int) -- + 18000 --Eastern time
select #todayepoch, #yesterdayepoch
select
*
from
tablename
where
epochdate between #yesterdayepoch and #todayepoch
I used UTC above as a presumption of comparing based on UTC times, but you could also compare to your time zone, with the appropriate addition/subtraction of your time zone difference in seconds (e.g., add 18000 to each variable to get noon in Eastern Standard Time).
You can test your results by using http://www.epochconverter.com/ to compare your values in your variables.
You query would look like the following:
DECLARE #dt_from DATETIME;
DECLARE #dt_to DATETIME;
SELECT
#dt_from=DATEADD(HH,-12,CAST(FLOOR(CAST(GETUTCDATE() AS FLOAT)) AS DATETIME)), -- strip time of current UTC date/time, and subtract 12 hrs
#dt_to=DATEADD(HH,+12,CAST(FLOOR(CAST(GETUTCDATE() AS FLOAT)) AS DATETIME)); -- strip time of current UTC date/time, and add 12 hrs
SELECT
*
FROM
tablename
WHERE
epochdate BETWEEN DATEDIFF(s,'1970-01-01',#dt_from) AND DATEDIFF(s,'1970-01-01',#dt_to);
When I execute the following, when the server's TimeZone is +01:00:
Convert(datetime, '2015-02-10T23:00:00Z', 127)
The result is:
10.02.2015 23:00:00
That is the Date at UTC-0. My expected value would be 11.02.2015 00:00:00, that is the date converted to the server's TimeZone.
Convert function doesn't convert time to UTC. It just simply changes the format of the input string. Here is what you need to do.
Find difference in hours between server local time and UTC time:
DECLARE #hour INT
SELECT #hour = DATEDIFF(HOUR, GETUTCDATE(), GETDATE())
Add the difference to the date you're trying to convert:
SELECT CONVERT(DATETIME, DATEADD(hour, #hour, '2015-02-10T23:00:00Z'), 127)
If you know your timezone difference in hours and you know that it's unlikely to be changed, then use a shorter version:
SELECT CONVERT(DATETIME, DATEADD(hour, -1, '2015-02-10T23:00:00Z'), 127)
You shouldn't really depend on the time zone setting of a server. However, if you have a specific time zone in mind, you could use my SQL Server Time Zone Support project.
After installation:
SELECT Tzdb.UtcToLocal('2015-02-10T23:00:00Z', 'Europe/Paris')
Choose a time zone from the list here.