I was trying to select a record between two date ranges but I keep getting duplicate record when two date range overlaps as shown below.
Here is an example.
Policy Info
Policy # Policy Effective Date Policy termination date Year
001 2018-10-01 2019-10-01 2018
002 2019-10-01 2020-10-01 2019
003 2020-10-01 2021-10-01 2020
004 2021-10-01 2022-10-01 2022
Policy Limit
LimitID Effective Date Termination Date Limit
1 2018-10-01 2021-10-01 1000
2 2018-10-01 3000-01-01 2500
How can I select Limit ID: 1 for Policy #: 001,002 003 or for the years 2018, 2019, 2020 and for any policy effective date greater than 2021-01-01 use Limit ID = 2
I tried the following but it keeps creating dupicate
((limit.effective_from_date < policy.effective_to_date
AND limit.effective_to_date > policy.effective_from_date
)
OR
(limit.effective_from_date = policy.effective_from_date
AND limit.effective_to_date = CONVERT(datetime, '01/01/3000', 102)))
but the above condition creates a duplicate. Is there any effective way of selecting a record within overlapping date ranges.
Any help will be appreciated!
Your problem is that you have overlapping periods for Policy Limits and you need to choose one. For what I understand from your data and I'm inferring a lot, you need to get the first limit for the FIRST period that it's [Policy Limit].[Effective Date] is earlier than the [Policy Info].[Policy Effective Date]
while [Policy Limit].[Termination Date] is later than [Policy Info].[Policy Termination Date].
If all my guessing is correct, you can do something like
drop table if exists #PolicyInfo
drop table if exists #PolicyLimit
CREATE TABLE #PolicyInfo (
Policy INT,
Policy_Effective_Date DATE,
Policy_termination_date DATE,
[Year] int
)
CREATE TABLE #PolicyLimit(
LimitID INT,
Effective_Date DATE,
Termination_Date DATE,
Limit INT
)
INSERT INTO #PolicyInfo (Policy, Policy_Effective_Date, Policy_termination_date, [Year])
VALUES
(001, '2018-10-01', '2019-10-01', 2018),
(002, '2019-10-01', '2020-10-01', 2019),
(003, '2020-10-01', '2021-10-01', 2020),
(004, '2021-10-01', '2022-10-01', 2022)
INSERT INTO #PolicyLimit (LimitID, Effective_Date, Termination_Date, Limit)
VALUES
(1, '2018-10-01','2021-10-01',1000),
(2, '2018-10-01','3000-01-01',2500)
;with cte AS (
-- Join PolicyInfo with PolicyLimit
-- condition: Policy_Effective_Date are between Effective_Date, pl.Termination_Date
-- AND
-- Policy_Termination_Date are between Effective_Date, pl.Termination_Date
SELECT *,
-- rank with partion by Policy
ROW_NUMBER() OVER (PARTITION BY [pi].Policy ORDER BY pl.Effective_Date, pl.Termination_Date) rn
FROM #PolicyInfo [pi]
INNER JOIN #PolicyLimit pl ON
[pi].Policy_Effective_Date BETWEEN pl.Effective_Date AND pl.Termination_Date
AND [pi].Policy_termination_date BETWEEN pl.Effective_Date AND pl.Termination_Date
)
SELECT Policy, LimitID
FROM cte
WHERE rn = 1 -- Select the first Limit per partition
I have an ID column, and a time column. I want to group the IDs by average time.
IDs: 1234, 1234, 5678, 5678
Times: 13:21, 19:55, 14:25, 15:04
select ID,
avg(cast(CONCAT(left(cast(Time as varchar),2),substring(cast(Time as varchar),4,2)) as int)*1.0)
It does return a result, but I don't believe the average to be correct as the average time can be outside of normal time constraints (aka the minutes can be > 59).
time stores a point in time, not a duration. What would you do for a duration longer than a day? You should instead store either the duration in seconds, minutes, what have you, and format it as hh:mm etc. when you want to display it. Or better yet, store a start date and end date, which is more complete information, and you can always derive the duration (in whatever format you like) from that.
Anyway, dealing with what you have, and assuming this table and sample data:
CREATE TABLE dbo.BadChoices
(
ID int,
DurationWithWrongType time(0)
);
INSERT dbo.BadChoices(ID, DurationWithWrongType) VALUES
(1234, '13:21'),
(1234, '19:55'),
(5678, '14:25'),
(5678, '15:04');
You could I suppose do:
SELECT ID, AvgDuration = CONVERT(DECIMAL(10,2),
AVG(DATEDIFF(MINUTE, '00:00', DurationWithWrongType)*1.0))
FROM dbo.BadChoices
GROUP BY ID;
Output:
ID
AvgDuration
1234
998.00
5678
884.50
Example db<>fiddle
If you want the display to be HH:MM, and you know for sure your durations will always be < 24 hours, you could do:
;WITH src AS
(
SELECT ID, AvgDuration = CONVERT(DECIMAL(10,2),
AVG(DATEDIFF(MINUTE, '00:00', DurationWithWrongType)*1.0))
FROM dbo.BadChoices
GROUP BY ID
)
SELECT ID, AvgDuration,
AvgDurHHMMSS = CONVERT(time(0), DATEADD(SECOND, AvgDuration*60, '00:00'))
FROM src;
Output:
ID
AvgDuration
AvgDurHHMMSS
1234
998.00
16:38:00
5678
884.50
14:44:30
Example db<>fiddle
We have to cast to datetime to be able to cast to float. We can then find the average and cast back to datetime and then back to time.
A second alternative is to convert the time into minutes, get the average and then use dateadd() and cast back to time
create table times(
t time);
insert into times values
('13:21'),
('19:55'),
('14:25'),
('15:04');
GO
4 rows affected
select
cast(
cast(
avg(
cast(
cast(t
as datetime)
as float)
)
as datetime)
as time)
from times
GO
| (No column name) |
| :--------------- |
| 15:41:15 |
select
cast(
dateadd(second,
avg(
DateDiff(second,0,t)
),
2000-01-01)
as time)
from times
GO
| (No column name) |
| :--------------- |
| 15:41:15 |
db<>fiddle here
The Postgres database claims it supports the ISO-8601 standard. In ISO-8601 a date format "yyyy", i.e. consisting of only the year, is fine and acceptable. But I can't find a way to add only a year to a Postgres database field of the type "date". Any idea if I'm doing something wrong or is this feature missing in Postgres?
I've seen other posts advising to set the date to "yyyy-01-01" but that is not what I want and need (since it marks a specific day of a month of a year).
Scenario
The scenario is the following. We are collecting information on people. For many we do have exact dates. But some have no dates or only years, or year and month but no day. We do have to be able to find people born before some year, or after some other year. This is easy if you have a full date. I hoped there would be some feature implemented in Postgres that would handle cases of incomplete dates.
To get the year of a date data type:
select extract(year from '2014-01-01'::date) as the_year;
the_year
----------
2014
If you only need the year then use a smallint with a check constraint
create table t (
the_year smallint check(
the_year between 0 and extract(year from current_date)
)
);
insert into t (the_year) values (2030);
ERROR: new row for relation "t" violates check constraint "t_the_year_check"
DETAIL: Failing row contains (2030).
insert into t (the_year) values (2014);
INSERT 0 1
But if you will store the whole date then it makes no sense to separate into 3 columns.
Note that the semantics of the column are up to the application. If a column is of the date type but the application only considers the year then that column means the year.
Check the Date/Time Functions and Operators
One solution to the partial date problem pointed by #a_horse in the comments is to create a column indicating the precision of that date
create table t (
the_date date,
the_date_precision varchar(5)
);
insert into t (the_date, the_date_precision) values
(current_date, 'year'),
(current_date, 'month'),
(current_date, 'day')
;
select
case the_date_precision
when 'year' then to_char(the_date, 'YYYY')
when 'month' then to_char(the_date, 'YYYY-MM')
else to_char(the_date, 'YYYY-MM-DD')
end as the_date
from t
;
the_date
------------
2014
2014-02
2014-02-06
The above is the KISS aproach but I think the next implementation is more elegant
create table t (
the_date date,
the_date_precision smallint
);
insert into t (the_date, the_date_precision) values
(current_date, 1),
(current_date, 2),
(current_date, 3)
;
select
array_to_string(
(
string_to_array(to_char(the_date, 'YYYY-MM-DD'), '-')
)[1:the_date_precision]
, '-'
) as the_date
from t
;
the_date
------------
2014
2014-02
2014-02-06
That select expression could be turned into a function to be easier to reproduce. Or just a view
create view view_t as
select *,
array_to_string(
(
string_to_array(to_char(the_date, 'YYYY-MM-DD'), '-')
)[1:the_date_precision]
, '-'
) as the_date_output
from t
;
select * from view_t;
the_date | the_date_precision | the_date_output
------------+--------------------+-----------------
2014-02-06 | 1 | 2014
2014-02-06 | 2 | 2014-02
2014-02-06 | 3 | 2014-02-06
I have a query using the DATEADD method which takes a lot of time.
I'll try to simplify what we do.
We are monitoring tempretures and every 5 minutes we store the highest temp and lowest temp in
table A
Date | Time | MaxTemp | MinTemp
2011-09-18 | 12:05:00 | 38.15 | 38.099
2011-09-18 | 12:10:00 | 38.20 | 38.10
2011-09-18 | 12:15:00 | 38.22 | 38.17
2011-09-18 | 12:20:00 | 38.21 | 38.20
...
2011-09-19 | 11:50:00 | 38.17 | 38.10
2011-09-19 | 12:55:00 | 38.32 | 38.27
2011-09-19 | 12:00:00 | 38.30 | 38.20
Date/Time columns are of type date/time (and not datetime)
In another table (Table B) we store some data for the entire day, where a day is from NOON (12PM) to noon (not midnight to midnight).
So table B columns include:
Date (date only no time)
ShiftManager
MaxTemp (this is the max temp for the entire 24 hours starting at that date noon till next day noon)
MinTemp
I get table B with all the data and just need to update the MaxTemp and MinTemp using table A
For example:For 09/18/2011 I need the maximum temp reading that was between 09/18/2011 12PM and 09/19/2011 12PM.
In the TableA sample we have above, the returend result would be 38.32 as it is the MAX(MaxTemp) for the desired period.
The SQL I'm using:
update TableB
set MaxTemp = (
select MAX(HighTemp) from TableA
where
(Date=TableB.Date and Time > '12:00:00')
or
(Date=DATEADD(dd,1,TableB.Date) and Time <= '12:00:00')
)
And it takes a lot of time (if I remove the DATEADD method it is quick).
Here is a simplified sample that shows the data I have and the expected result:
DECLARE #TableA TABLE ([Date] DATE, [Time] TIME(0), HighTemp DECIMAL(6,2));
DECLARE #TableB TABLE ([Date] DATE, MaxTemp DECIMAL(6,2));
INSERT #TableA VALUES
('2011-09-18','12:05:00',38.15),
('2011-09-18','12:10:00',38.20),
('2011-09-18','12:15:00',38.22),
('2011-09-19','11:50:00',38.17),
('2011-09-19','11:55:00',38.32),
('2011-09-19','12:00:00',38.31),
('2011-09-19','12:05:00',38.33),
('2011-09-19','12:10:00',38.40),
('2011-09-19','12:15:00',38.12),
('2011-09-20','11:50:00',38.27),
('2011-09-20','11:55:00',38.42),
('2011-09-20','12:00:00',38.16);
INSERT #TableB VALUES
('2011-09-18', 0),
('2011-09-19', 0);
-- This is how I get the data, now I just need to update the max temp for each day
with TableB(d, maxt) as
(
select * from #TableB
)
update TableB
set maxt = (
select MAX(HighTemp) from #TableA
where
(Date=TableB.d and Time > '12:00:00')
or
(Date=DATEADD(dd,1,TableB.d) and Time <= '12:00:00')
)
select * from #TableB
Hope I was able to explian myself, any ideas how can I do it differently? Thx!
Functions on column usually kill performance. So can OR.
However, I assume you want AND not OR because it is a range.
So, applying some logic and having just one calculation
update TableB
set MaxTemp =
(
select MAX(HighTemp) from TableA
where
(Date + Time - 0.5 = TableB.Date)
)
(Date + Time - 0.5) will change noon to noon to be midnight to midnight (0.5 = 12 hours). More importantly, you can make this a computed column and index it
More correctly, Date + Time - 0.5 is DATEADD(hour, -12, Date+Time) assuming Date and Time are real dates/times and not varchar...
Edit: this answer is wrong but I'll leave it up as "what not to do"
See this for more:
Bad Habits to Kick : Using shorthand with date/time operations
This would probably be a lot easier if you used a single SMALLDATETIME column instead of separating this data into DATE/TIME columns. Also I'm assuming you are using SQL Server 2008 and not a previous version where you're storing DATE/TIME data as strings. Please specify the version of SQL Server and the actual data types being used.
DECLARE #d TABLE ([Date] DATE, [Time] TIME(0), MaxTemp DECIMAL(6,3), MinTemp DECIMAL(6,3));
INSERT #d VALUES
('2011-09-18','12:05:00',38.15,38.099),
('2011-09-18','12:10:00',38.20,38.10),
('2011-09-18','12:15:00',38.22,38.17),
('2011-09-18','12:20:00',38.21,38.20),
('2011-09-19','11:50:00',38.17,38.10),
('2011-09-19','12:55:00',38.32,38.27),
('2011-09-19','12:00:00',38.30,38.20);
SELECT '-- before update';
SELECT * FROM #d;
;WITH d(d,t,dtr,maxt) AS
(
SELECT [Date], [Time], DATEADD(HOUR, -12, CONVERT(SMALLDATETIME, CONVERT(CHAR(8),
[Date], 112) + ' ' + CONVERT(CHAR(8), [Time], 108))), MaxTemp FROM #d
),
d2(dtr, maxt) AS
(
SELECT CONVERT([Date], dtr), MAX(maxt) FROM d
GROUP BY CONVERT([Date], dtr)
)
UPDATE d SET maxt = d2.maxt FROM d
INNER JOIN d2 ON d.dtr >= d2.dtr AND d.dtr < DATEADD(DAY, 1, d2.dtr);
SELECT '-- after update';
SELECT * FROM #d;
Results:
-- before update
2011-09-18 12:05:00 38.150 38.099
2011-09-18 12:10:00 38.200 38.100
2011-09-18 12:15:00 38.220 38.170
2011-09-18 12:20:00 38.210 38.200
2011-09-19 11:50:00 38.170 38.100
2011-09-19 12:55:00 38.320 38.270
2011-09-19 12:00:00 38.300 38.200
-- after update
2011-09-18 12:05:00 38.220 38.099
2011-09-18 12:10:00 38.220 38.100
2011-09-18 12:15:00 38.220 38.170
2011-09-18 12:20:00 38.220 38.200
2011-09-19 11:50:00 38.220 38.100
2011-09-19 12:55:00 38.320 38.270
2011-09-19 12:00:00 38.320 38.200
Presumably you want to update the MinTemp as well, and that would just be:
;WITH d(d,t,dtr,maxt,mint) AS
(
SELECT [Date], [Time], DATEADD(HOUR, -12,
CONVERT(SMALLDATETIME, CONVERT(CHAR(8), [Date], 112)
+ ' ' + CONVERT(CHAR(8), [Time], 108))), MaxTemp, MaxTemp
FROM #d
),
d2(dtr, maxt, mint) AS
(
SELECT CONVERT([Date], dtr), MAX(maxt), MIN(mint) FROM d
GROUP BY CONVERT([Date], dtr)
)
UPDATE d
SET maxt = d2.maxt, mint = d2.maxt
FROM d
INNER JOIN d2
ON d.dtr >= d2.dtr
AND d.dtr < DATEADD(DAY, 1, d2.dtr);
Now, this is not really better than your existing query, because it's still going to be using scans to figure out aggregates and all the rows that need to be updating. I'm not saying you should be updating the table at all, because this information can always be derived at query time, but if it is something you really want to do, I would combine the advice in these answers and consider revising the schema. For example, if the schema were:
USE [tempdb];
GO
CREATE TABLE dbo.d
(
[Date] SMALLDATETIME,
MaxTemp DECIMAL(6,3),
MinTemp DECIMAL(6,3),
RoundedDate AS (CONVERT(DATE, DATEADD(HOUR, -12, [Date]))) PERSISTED
);
CREATE INDEX rd ON dbo.d(RoundedDate);
INSERT dbo.d([Date],MaxTemp,MinTemp) VALUES
('2011-09-18 12:05:00',38.15,38.099),
('2011-09-18 12:10:00',38.20,38.10),
('2011-09-18 12:15:00',38.22,38.17),
('2011-09-18 12:20:00',38.21,38.20),
('2011-09-19 11:50:00',38.17,38.10),
('2011-09-19 12:55:00',38.32,38.27),
('2011-09-19 12:00:00',38.30,38.20);
Then your update is this simple, and the plan is much nicer:
;WITH g(RoundedDate,MaxTemp)
AS
(
SELECT RoundedDate, MAX(MaxTemp)
FROM dbo.d
GROUP BY RoundedDate
)
UPDATE d
SET MaxTemp = g.MaxTemp
FROM dbo.d AS d
INNER JOIN g
ON d.RoundedDate = g.RoundedDate;
Finally, one of the reasons your existing query is probably taking so long is that you are updating all of time, every time. Is data from last week changing? Probably not. So why not limit the WHERE clause to recent data only? I see no need to go recalculate anything earlier than yesterday unless you are constantly receiving revised estimates of how warm it was last Tuesday at noon. So why are there no WHERE clauses on your current query, to limit the date range where it is attempting to do this work? Do you really want to update the WHOLE able, EVERY time? This is probably something you should only be doing once a day, sometime in the afternoon, to update yesterday. So whether it takes 2 seconds or 2.5 seconds shouldn't really matter.
You may need to use -12 depending on date as start date or end date for the noon to noon internal.
update tableA
set tableAx.MaxTemp = MAX(TableB.HighTemp)
from tableA as tableAx
join TableB
on tableAx.Date = CAST(DATEADD(hh,12,TableB.[Date]+TableB.[Time]) as Date)
group by tableAx.Date
Because of the 12 hour offset not sure how much would would gain by putting TableB Date plus Time in a DateTime field directly. Cannot get away from the DATEADD and the output from a functions is not indexed even if the parameters going into the function are indexed. What you might be able to to is create a computed column that = date + time +/- 12h and index that column.
Like the recommendation from Arron to only update those without values.
update tableA
set tableAx.MaxTemp = MAX(TableB.HighTemp)
from tableA as tableAx
join TableB
on tableAx.Date = CAST(DATEADD(hh,12,TableB.[Date]+TableB.[Time]) as Date)
where tableAx.MaxTemp is null
group by tableAx.Date
or an insert of new dates
insert into tableA (date, MaxTemp)
select CAST(DATEADD(hh,12,TableB.[Date]+TableB.[Time]), as Date) as [date] , MAX(TableB.HighTemp) as [MaxTemp]
from tableA as tableAx
right outer join TableB
on tableAx.Date = CAST(DATEADD(hh,12,TableB.[Date]+TableB.[Time]) as Date)
where TableB.Date is null
group by CAST(DATEADD(hh,12,TableB.[Date]+TableB.[Time]) as Date)
I have a table like below
Stage 1
Table Name : Product
Date_From Date_To Available
01/03/2011 05/03/2011 5
06/03/2011 15/03/2011 6
Stage 2
If I update above table with below data
Date_From Date_To Available
04/03/2011 08/03/2011 4
10/03/2011 18/03/2011 2
Stage 3
I need output of Product table (Stage 1) like this, when update with stage 2 data at same time
Date_From Date_To Available
01/03/2011 03/03/2011 5
04/03/2011 08/03/2011 4
06/03/2011 15/03/2011 6
10/03/2011 18/03/2011 2
Hoping for your help
EDIT:
create table t (dt_from datetime,dt_to datetime, Available int)
insert into t values ('20110301','20110305',5)
insert into t values ('20110306','20110315',6)
run after trigger is created
--insert into t values ('20110304','20110308',4)
---insert into t values ('20110310','20110318',2)
SELECT *,ROW_NUMBER() OVER (ORDER BY dt_from) rn FROM t
CREATE TRIGGER my_tr ON t FOR INSERT
AS
UPDATE t SET dt_to=(SELECT TOP 1 DATEADD(d,-1,dt_from) FROM inserted t1 WHERE dt_from
BETWEEN t.dt_from AND t.dt_to AND t.Available<>t1.Available)
WHERE EXISTS (SELECT * FROM inserted t1 WHERE dt_from
BETWEEN t.dt_from AND t.dt_to AND t.Available<>t1.Available)
UPDATE t SET dt_from=(SELECT TOP 1 DATEADD(d,1,dt_to) FROM inserted t1 WHERE dt_to
BETWEEN t.dt_from AND t.dt_to AND t.Available<>t1.Available)
WHERE EXISTS (SELECT * FROM inserted t1 WHERE dt_to
BETWEEN t.dt_from AND t.dt_to AND t.Available<>t1.Available)
the above code is working fine , i get the below result
Date_From Date_To Available
01/03/2011 03/03/2011 5
04/03/2011 08/03/2011 4
09/03/2011 09/03/2011 6
10/03/2011 18/03/2011 2
But only one problem i facing is
insert into t values ('20110301','20110318',5)
it not get the correct result
i need result like below
Date_From Date_To Available
01/03/2011 18/03/2011 5
Hoping your help
You should probably have a stored procedure that takes your values, and then decides whether an INSERT or UPDATE is needed, based on the dates...
CREATE PROCEDURE dbo.InsertOrUpdateData
#FromDate DATE, #ToDate DATE, #NewAvail INT
AS
IF EXISTS(SELECT * FROM dbo.YourTable
WHERE Date_From = #FromDate AND Date_To = #ToDate)
UPDATE dbo.YourTable
SET Available = Available + #NewAvail
WHERE Date_From = #FromDate AND Date_To = #ToDate
ELSE
INSERT INTO dbo.YourTable(Date_From, Date_To, Availability)
VALUES(#FromDate, #ToDate, #NewAvail)
With this stored procedure, you can call
EXEC dbo.InsertOrUpdateData
#FromDate = '20110301', #ToDate = '20110305', #NewAvail = 42
If those dates already exists, that row will be updated, if those dates don't exist, a new row will be inserted. No trigger voodoo or anything like that needed....
Is that what you're looking for??
When you get a new row you need to do the following:
Using the new ValidFrom and ValidTo dates - Checking existing data for anything that either:
Has a ValidFrom < new Validfrom and ValidTo >= new ValidFrom - if found, ValidTo should be set to 1 day before new ValidFrom.
Has a ValidFrom >= new ValidFrom and ValidTo <= new ValidTo - if found, You might need to do check again to see if moving the dates would overlap previous entries
Has a ValidFrom >= new ValidFrom and ValidFrom <= new ValidTo and ValidTo >= new ValidTo. This record might need to be split in two.
etc - any other concievable combination of Date overlaps ...
I would rather recommend using a new table with day - available, giving one row per product per day, with the available measure. This way, any new entries with validfrom / validto can just write the available value for the relevant days for the relevant products.
DayID ProductId Available