I have to query an SQL Server database and the table's values use Epoch time (an int. Here's an example - 1438005018). I am wondering how I can write a query so that I can say the following...
select
*
from
tablename
where
epochdate between 'yesterday at 12:00' and 'today at 12:00' --this is the part I'm not sure about.
Ideally, if it's easy, I'd like the query to use non-epoch logic as Epoch time confuses the crap out of me. Maybe there's a quick way of converting in SQL Server?
I posted a link above in the comments that may be a more practical solution if you're able to deploy functions in the database you're working with, but if you're only able to query, this is an option to try as well (this assumes SQL Server 2008 and above):
declare #todayepoch bigint, #yesterdayepoch bigint;
select #todayepoch =
cast((cast(dateadd(hour, 12,
cast(cast(sysutcdatetime() as date) as datetime)) as decimal(24,10))
- cast(cast('1970-01-01' as datetime) as decimal(24,10)))
*60.0*60.0*24.0 as int), -- + 18000, --Eastern time
#yesterdayepoch =
cast((cast(dateadd(hour, -12,
cast(cast(sysutcdatetime() as date) as datetime)) as decimal(24,10))
- cast(cast('1970-01-01' as datetime) as decimal(24,10)))
*60.0*60.0*24.0 as int) -- + 18000 --Eastern time
select #todayepoch, #yesterdayepoch
select
*
from
tablename
where
epochdate between #yesterdayepoch and #todayepoch
I used UTC above as a presumption of comparing based on UTC times, but you could also compare to your time zone, with the appropriate addition/subtraction of your time zone difference in seconds (e.g., add 18000 to each variable to get noon in Eastern Standard Time).
You can test your results by using http://www.epochconverter.com/ to compare your values in your variables.
You query would look like the following:
DECLARE #dt_from DATETIME;
DECLARE #dt_to DATETIME;
SELECT
#dt_from=DATEADD(HH,-12,CAST(FLOOR(CAST(GETUTCDATE() AS FLOAT)) AS DATETIME)), -- strip time of current UTC date/time, and subtract 12 hrs
#dt_to=DATEADD(HH,+12,CAST(FLOOR(CAST(GETUTCDATE() AS FLOAT)) AS DATETIME)); -- strip time of current UTC date/time, and add 12 hrs
SELECT
*
FROM
tablename
WHERE
epochdate BETWEEN DATEDIFF(s,'1970-01-01',#dt_from) AND DATEDIFF(s,'1970-01-01',#dt_to);
Related
We transfer data from Oracle SQL to SQL Server in the near real-time data warehouse. (Oracle is a transactional database) The time changes twice a year for the data we use. We have been having issues when calculating DateTime differences and I have to work on a short-term and long-term solutions where I need advice.
New data coming in :
I can create tables from scratch and use datetimeoffset data field for all the time variables. Would this
display dates in SQL Server queries in the local time zone and save them in their actual UTC time? And when I
calculate the differences, would it give the correct difference?
Existing data without offset information:
I already have data converted into the current timezone. We have 2 timezones AEST and AEDT.
When we calculate time differences for KPIs it gives incorrect times. Any ideas on how it can be resolved?
Can creating a function that converts the dates and times into UTC and subtracts the difference and returns the value would work?
An example below :
DECLARE #startdate DateTime = '2022-10-01 23:13:00.000'; --UTC 2022-10-01 13:13:00.00
DECLARE #enddate DateTime = '2022-10-02 12:08:00.000' --UTC 2022-10-02 01:08:00.00
select CAST((#enddate - #startdate) as time(0)) 'Difference'
Time difference: 12:55:00
Actual Time Difference: 11:55:00
For your example, if you include the time zone info with the data, the date calculations work out
DECLARE #startdate datetimeoffset(0) = '2022-10-01 23:13:00 +10:00'; --UTC 2022-10-01 13:13:00.00
DECLARE #enddate datetimeoffset(0) = '2022-10-02 12:08:00 +11:00' --UTC 2022-10-02 01:08:00.00
declare #minutes int = datediff(minute, #startdate, #enddate);
select #startdate at time zone 'utc',
#enddate at time zone 'utc';
select [hours] = #minutes / 60,
[minutes] = #minutes - 60 * (#minutes / 60)
To address the existing data, you can do a one-time conversion to datetimeoffset by running it through at time zone with the appropriate time zone name. In my example above, I used UTC but you can find all of the supported time zones in sys.time_zone_info.
As to your first question about the internal storage format and display, I'm not sure if it uses UTC internally or not. As to display, it will, by default, output them with the time zone that you stored them with. That is, if you stored a given value with +10:00 as the offset, that's how it will display. You can convert via methods already described above or with switchoffset(). Lastly, if it matters, you can get just the offset with datepart noting that it returns the offset in number of minutes (to conform with datepart needing to return an int).
Here's what I use to get a HH:MM string:
CONVERT(varchar(5), time_requested, 108)
I'm thinking there may be a more elegant way, even maybe more efficient (see similar question on How to remove the time portion of a datetime value (SQL Server)?).
Requirements:
the final result has to be easily convertible to a string (in order to be able to concatenate some field time_created with a field such as date_created).
the following 2 cases must be covered: HH:MM and HH:MM:SS.
Declare #D datetime = GetDate() -- Data Type could be Time as well
Select CONVERT(varchar(8), #D, 108) -- for HH:MM:SS 11:27:26
Select CONVERT(varchar(5), #D, 108) -- for HH:MM 11:27
Failed to mention, if 2012+ you could also use Format()
Declare #D DateTime = '2016-10-22 13:30:25'
Select Format(#D,'HH:mm') -- 13:30
Select Format(#D,'hh:mm:ss tt') -- 01:30:25 PM
Your method is fine. It produces a string representation of the value.
You can also convert to a time data type:
select cast(time_requested as time)
But note that this is "format-less" -- it contains hours, minutes, seconds, and fractions of seconds. So, your method is probably the better approach.
I have an SSRS Date/Time parameter generated from a shared dataset query against a SQL Server datetime field. The parameter displays correctly in a report textbox but it will not work in an embedded dataset query, even against the same table that the datetime value was generated from.
In order to use the parameter for a dataset query I have to parse both sides of a where clause to get it to work in Preview in SSDT:
(convert(varchar,invoice.snapshot_datetime,120)) = (convert(varchar,#snapshotdatetime,120))
This is tremendously inefficient.
How can I get my where clause to work without parsing the invoice.snapshot_datetime column?
Server Details
The SQL Server Language is English (United States).
SQL Server dateformat is mdy (from dbcc useroptions).
Getdate() returns '2015-05-20 10:27:56.687' in SSMS
Assuming your date range is between 1900-01-01 and 2079-06-06 you can cast to SmallDateTime to truncate the seconds out of your datetime variable:
DECLARE #DateTime datetime
SET #DateTime = CAST(CAST(#snapshotdatetime as SmallDateTime) as DateTime)
(thanks to t-clausen.dk for his answer here)
Now, since your actual column is of type DateTime, it does keep seconds (and milliseconds), and you will need to eliminate them as well.
However, using functions on your column will prevent the SQL Server from using any indexes you might have on this column, so a better approach would be to use a DateTime range:
DECLARE #FromDateTime datetime, #ToDateTime datetime
SET #FromDateTime = CAST(CAST(#snapshotdatetime as SmallDateTime) as DateTime)
Since the cast will round the minutes of the small date time up if it's over 29.998 seconds, and down if it's below 29.999 seconds. You always want to round down since it's From datetime, you need to cheke if you need to decrease a minute:
IF datepart(second, #snapshotdatetime) > 29
OR (datepart(second, #snapshotdatetime) = 29
AND datepart(millisecond, #snapshotdatetime) > 998)
SET #FromDateTime = DATEADD(minute, -1, #FromDateTime)
SET #ToDateTime = DATEADD(minute, 1, #FromDateTime)
and then, in your where clause, use this:
invoice.snapshot_datetime <= #FromDateTime
AND invoice.snapshot_datetime >= #ToDateTime
If you haven't found solution yet, try this:
select (convert(varchar,GETDATE(),112))
it will return 20180206 (yyymmdd)
I have the following query :
select CONVERT(varchar(12), DATEADD(MILLISECOND, DateDiff(MILLISECOND, '2014-08-04 10:37:28.713','2014-11-04 08:21:17.723'), 0), 114)
When I execute this, I get the error :
"The datediff function resulted in an overflow. The number of dateparts separating two date/time instances is too large. Try to use datediff with a less precise datepart."
When I change the query to the following it works fine :
select CONVERT(varchar(12), DATEADD(SECOND, DateDiff(SECOND, '2014-08-04 10:37:28.713','2014-11-04 08:21:17.723'), 0), 114)
The problem is that I really need the MILLISECONDS as well.
A bit later response but may help.
In SQL 2016 MS introduced function DATEDIFF_BIG which will (according to type size) overflow in difference bigger than something like 290k years. But technet article have same time difference as basic DATEDIFF - https://msdn.microsoft.com/en-us/library/mt628058.aspx
See https://learn.microsoft.com/en-us/sql/t-sql/functions/datediff-transact-sql?view=sql-server-ver15#return-value
For millisecond, the maximum difference between startdate and enddate is 24 days, 20 hours, 31 minutes and 23.647 seconds.
If you need millisecond above that level, you'll need to write something custom.
In SQL Server 2016 there is a new function available: DATEDIFF_BIG
It solves exactly the overflow problem.
You don't need to refer to the miliseconds in your calculation.
This will do exactly the same as your script except the overflow:
SELECT CONVERT(varchar(12),
CAST('2014-11-04 08:21:17.723' as datetime) -
CAST('2014-08-04 10:37:28.713' as datetime)
, 114)
For me there was a big interval between two dates so i have used below code
declare #timetagInMillsecond bigint=CAST(CAST( cast(#timetag as
datetime) -'1970-01-01' AS decimal(38,10))*24*60*60*1000+0.5 as
bigint)
It works for me .
Use DATEDIFF_BIG to resolve the overflow issue
The datediff function resulted in an overflow. The number of dateparts separating two date/time instances is too large. Try to use datediff with a less precise datepart.
SELECT DATEDIFF_BIG(
millisecond,
SYSDATETIME(),
DATEADD(year, 1000, SYSDATETIME()) ) AS 'Milliseconds in 1000 years';
For SQL Server 2014, the following works around the 'int' limitation to obtain a "JavaScript Time Epoch". This assumes the start epoch is itself a date, which fits the local use-case leading to finding this question. The query requires adaptation to the specific question use-case which does not have this property.
declare #x as datetime = getdate()
-- epoch_delta_s_to_date * 1000 + day_delta_ms
select
cast(datediff(second, '1970-01-01', cast(#x as date)) as bigint) * 1000
+ cast(datediff(millisecond, cast(#x as date), #x) as bigint)
For the case of obtaining a "JavaScript Time Epoch" this is still subject to the Y2038 limitation of datediff(second, '1970-01-01', ..).
I've got a varchar field in SQL Sever 2005 that's storing a time value in the format "hh:mm"ss.mmmm".
What I really want to do is take the average using the built in aggregate function of those time values. However, this:
SELECT AVG(TimeField) FROM TableWithTimeValues
doesn't work, since (of course) SQL won't average varchars. However, this
SELECT AVG(CAST(TimeField as datetime)) FROM TableWithTimeValues
also doesn't work. As near as I can tell, SQL doesn't know how to convert a value with only time and no date into a datetime field. I've tried a wide variety of things to get SQL to turn that field into a datetime, but so far, no luck.
Can anyone suggest a better way?
SQL Server can convert a time-only portion of a datetime value from string to datetime, however in your example, you have a precision of 4 decimal places. SQL Server 2005 only recognizes 3 places. Therefore, you will need to truncate the right-most character:
create table #TableWithTimeValues
(
TimeField varchar(13) not null
)
insert into #TableWithTimeValues
select '04:00:00.0000'
union all
select '05:00:00.0000'
union all
select '06:00:00.0000'
SELECT CAST(TimeField as datetime) FROM #TableWithTimeValues
--Msg 241, Level 16, State 1, Line 1
--Conversion failed when converting datetime from character string.
SELECT CAST(LEFT(TimeField, 12) as datetime) FROM #TableWithTimeValues
--Success!
This will convert valid values into a DATETIME starting on 1900-01-01. SQL Server calculates dates based on 1 day = 1 (integer). Portions of days are then portions of the value 1 (i.e. noon is 0.5). Because a date was not specified in the conversion, SQL Server assigned the value of 0 days (1900-01-01), which accommodates our need to average the time portion.
To perform an AVG operation on a DATETIME, you must first convert the DATETIME to a decimal value, perform the aggregation, then cast back. For example
SELECT CAST(AVG(CAST(CAST(LEFT(TimeField, 12) as datetime) AS FLOAT)) AS DATETIME) FROM #TableWithTimeValues
--1900-01-01 05:00:00.000
If you need to store this with an extra decimal place, you can convert the DATETIME to a VARCHAR with time portion only and pad the string back to 13 characters:
SELECT CONVERT(VARCHAR, CAST(AVG(CAST(CAST(LEFT(TimeField, 12) as datetime) AS FLOAT)) AS DATETIME), 114) + '0' FROM #TableWithTimeValues
Try this
AVG(CAST(CAST('1900-01-01 ' + TimeField AS DateTime) AS Float))
You really should store those in a datetime column anyway. Just use a consistent date for that part (1/1/1900 is very common). Then you can just call AVG() and not worry about it.
I used Cadaeic's response to get an answer I was looking for, so I thought I should share the code....
I was looking for a query that would average ALL my times together and give me an overall Turn Around Time for all approvals. Below is a nested statement that gives you the AVG TAT for individual id's and and when nested an overall TAT
SELECT
-- calculates overall TAT for ALL Approvals for specified period of time
-- depending on parameters of query
CONVERT(VARCHAR, CAST(AVG(CAST(CAST(LEFT(Tat_mins, 12) as datetime) AS FLOAT)) AS DATETIME), 108) + '0'
from
(
-- tat is for individual approvals
SELECT
dbo.credit_application.decision_status,
dbo.credit_application.application_id,
cast(dbo.credit_application.data_entry_complete as date) as'Data Entry Date',
cast(dbo.credit_application.decision_date as DATE) as 'Decision Date',
avg(datediff(minute, dbo.credit_application.data_entry_complete, dbo.credit_application.decision_date)) as 'TAT Minutes',
convert (char(5), DateAdd(minute, Datediff(minute,dbo.credit_application.data_entry_complete, dbo.credit_application.decision_date),'00:00:00'),108) as 'TAT_Mins'
FROM dbo.credit_application
where Decision_status not in ('P','N')
group by dbo.credit_application.decision_status,
dbo.credit_application.data_entry_complete,
dbo.credit_application.decision_date
--dbo.credit_application.application_id
)bb
How do you think to average on datetime?
I guess that you need to GROUP BY some period (Hour?), and display Count(*)?
SQL Server stores datetime data as 2 4-byte integers, hence a datetime take 8 bytes. The first is days since the base date and the second is milliseconds since midnight.
You can convert a datetime value to an integer and perform mathematical operations, but the convert only returns the "days" portion of the datetime value e.g. select convert(int,getdate()). It is more difficult to return the "time" portion as an integer.
Is using SQL Server 2008 an option for you? That version has a new dedicated time data type.
Thanks, Andy.
I'd work out the difference between all of the dates and an arbitrary point (01/01/1900), average it and then add it back on to the arbitrary point.