No doubt a similar question has come up before, but I haven't been able to locate it by searching...
I have a raw dataset with time series data including 'from' and 'to' date fields.
The problem is, when data is loaded, new records have been created ('to' date added to old record, new record 'from' load date) even where no values have changed.
I want to convert this to a table which just shows a row for each genuine change - and the from/ to dates reflecting this.
By way of example, the source data looks like this:
ID
Col1
Col2
Col3
From
To
Test1
1
1
1
01/01/2020
31/12/9999
Test2
1
2
3
01/01/2020
30/06/2020
Test2
1
2
3
01/07/2020
30/09/2020
Test2
3
2
1
01/10/2020
31/12/9999
The first two records for Test2 (rows 2 and 3) are essentially the same - there was no change when the second row was loaded on 01/07/2020. I want a single row for the period 01/01/2020 - 30/09/2020 for which there was no change:
ID
Col1
Col2
Col3
From
To
Test1
1
1
1
01/01/2020
31/12/9999
Test2
1
2
3
01/01/2020
30/09/2020
Test2
3
2
1
01/10/2020
31/12/9999
For this simplified example, I can achieve that by grouping by each column (apart from dates) and using the MIN from date/ MAX end date:
SELECT
ID, Col1, Col2, Col3, MIN(From) AS From, MAX(To) as TO
FROM TABLE
GROUP BY ID, Col1, Col2, Col3
However, this won't work if a value changes then subsequently changes back to what it was before eg
ID
Col1
Col2
Col3
From
To
Test1
1
1
1
01/01/2020
31/12/9999
Test2
1
2
3
01/01/2020
30/04/2020
Test2
1
2
3
01/05/2020
30/06/2020
Test2
3
2
1
01/07/2020
30/10/2020
Test2
1
2
3
01/11/2020
31/12/9999
Simply using MIN/ MAX in the code above would return this - so it looks like both sets of values were valid for the period from 01/07/2020 - 30/10/2020:
ID
Col1
Col2
Col3
From
To
Test1
1
1
1
01/01/2020
31/12/9999
Test2
1
2
3
01/01/2020
31/12/9999
Test2
3
2
1
01/07/2020
30/10/2020
Whereas actually the first set of values were valid before and after that period, but not during.
It should return a single row for instead of two for the period from 01/01/2020 - 30/06/2020 when there were no changes for this ID, but then another row for the period when the values were different, and then another row where it reverted to the initial values, but with a new From date.
ID
Col1
Col2
Col3
From
To
Test1
1
1
1
01/01/2020
31/12/9999
Test2
1
2
3
01/01/2020
30/06/2020
Test2
3
2
1
01/07/2020
30/10/2020
Test2
1
2
3
01/11/2020
31/12/9999
I'm struggling to conceptualise how to approach this.
I'm guessing I need to use LAG somehow but not sure how to apply it - eg rank everything in a staging table first, then use LAG to compare a concatenation of the whole row?
I'm sure I could find a fudged way eventually, but I've no doubt this problem has been solved many times before so hoping somebody can point me to a simpler/ neater solution than I'd inevitably come up with...
Advanced Gaps and Islands
I believe this is an advanced "gaps and islands" problem. Use that as a search term and you'll find plenty of literature on the subject. Only difference is normally only one column is being tracked, but you have 3.
No Gaps Assumption
One major assumption of this script is there is no gap in the overlapping dates, or in other words, it assumes the previous rows ToDate = current FromDate - 1 day.
Not sure if you need to account for gaps, would be simple just add criteria to IsChanged to check for that
Multi-Column Gaps and Islands Solution
DROP TABLE IF EXISTS #Grouping
DROP TABLE IF EXISTS #Test
CREATE TABLE #Test (ID INT IDENTITY(1,1),TestName Varchar(10),Col1 INT,Col2 INT,Col3 INT,FromDate Date,ToDate DATE)
INSERT INTO #Test VALUES
('Test1',1,1,1,'2020-01-01','9999-12-31')
,('Test2',1,2,3,'2020-01-01','2020-04-30')
,('Test2',1,2,3,'2020-05-01','2020-06-30')
,('Test2',3,2,1,'2020-07-01','2020-10-30')
,('Test2',1,2,3,'2020-11-01','9999-12-31')
;WITH cte_Prev AS (
SELECT *
,PrevCol1 = LAG(Col1) OVER (PARTITION BY TestName ORDER BY FromDate)
,PrevCol2 = LAG(Col2) OVER (PARTITION BY TestName ORDER BY FromDate)
,PrevCol3 = LAG(Col3) OVER (PARTITION BY TestName ORDER BY FromDate)
FROM #Test
), cte_Compare AS (
SELECT *
,IsChanged = CASE
WHEN Col1 = PrevCol1
AND Col2 = PrevCol2
AND Col3 = PrevCol3
THEN 0 /*No change*/
ELSE 1 /*Iterate so new group created */
END
FROM cte_Prev
)
SELECT *,GroupID = SUM(IsChanged) OVER (PARTITION BY TestName ORDER BY ID)
INTO #Grouping
FROM cte_Compare
/*Raw unformatted data so you can see how it works*/
SELECT *
FROM #Grouping
/*Aggregated results*/
SELECT GroupID,TestName,Col1,Col2,Col3
,FromDate = MIN(FromDate)
,ToDate = MAX(ToDate)
,NumberOfRowsCollapsedIntoOneRow = COUNT(*)
FROM #Grouping
GROUP BY GroupID,TestName,Col1,Col2,Col3
How to select the value from the table based on category_id?
I have a table like this. Please help me.
Table A
ID Name category_id
-------------------
1 A 1
2 A 1
3 B 1
4 C 2
5 C 2
6 D 2
7 E 3
8 E 3
9 F 3
How to get the below mentioned output from table A?
ID Name category_id
--------------------
1 A 1
2 A 1
4 C 2
5 C 2
7 E 3
8 E 3
Give a row number for each row based on group by category_id and sort by ascending order of ID. Then select the rows having row number 1 and 2.
Query
;with cte as (
select [rn] = row_number() over(
partition by [category_id]
order by [ID]
), *
from [your_table_name]
)
select [ID], [Name], [category_id]
from cte
where [rn] < 3;
Kindly run this query It really help You Out.
SELECT tbl.id,tbl.name, tbl.category_id FROM TableA as tbl WHERE
tbl.name IN(SELECT tbl2.name FROM TableA tbl2 GROUP BY tbl2.name HAVING Count(tbl2.name)> 1)
Code select all category_id from TableA which has Name entries more then one. If there is single entry of any name group by category_id then such data will be excluded. In above example questioner want to eliminate those records that have single Name entity like wise category_id 1 has name entries A and B among which A has two entries and B has single entry so he want to eliminate B from result set.
I want to Update only top 1 OR only 1 row of a column where a column values are same.
(Just logical explanation don't go on syntax)
LIKE:
Update [Total] = (value from a another table with a common column)
but need to update only top 1 row OR any one row to the current (updating) table not all rows matching column value...
e.g
Table 1:
Skill Value
abc 3
def 4
xyz 3.5
Table 2:
Name Skill MyValue MyValue2(ColumnNeedsToBeUpdated)
Ram abc 3
shyam abc 4
Mohan abc 5
Raju xyz 4
Ratan xyz 6
Now I want to Update MyValue2 based on Table1 column Skill Value = MyValue2 but I want to update anyone OR top 1 row in Table2 NOT ALL Please help
Expected Output:
Name Skill MyValue MyValue2(ColumnNeedsToBeUpdated)
Ram abc 3 3
shyam abc 4
Mohan abc 5
Raju xyz 4 3.5
Ratan xyz 6
OR Alternate output can be:
Name Skill MyValue MyValue2(ColumnNeedsToBeUpdated)
Ram abc 3 Value from Table1 / no. of records with skill abc (3/3)
shyam abc 4
Mohan abc 5
Raju xyz 4 Value from Table1 / no. of records with skill xyz (3.5/2)
Ratan xyz 6
In Table 2, give a row number based on group by Skill column and order by MyValue column. And then updated the rows which having row number1 with Value from Table 1.
Query
;with cte as(
select [rn] = row_number() over(
partition by Skill
order by [MyValue]
), *
from [Table2]
)
update t1
set t1.[MyValue2] = t2.[Value]
from cte t1
join [Table1] t2
on t1.[Skill] = t2.[Skill]
where t1.[rn] = 1;
I'm sure this is simple, but it's been a rough day:
I have a table of data like below. I have a list of attribute IDs and need to get the distinct ProductID that matches all the AttributeIDs selected.
ProductID AttributeID
1 2
1 3
1 5
2 2
2 3
2 7
As an example, I want the distinct product ID that matches attributes 2, 3, AND 5.
What's the most optimal way to do this?
using count(distinct ) = n where n is the number of elements in the where AttributeId in () list.
select ProductId
from t
where AttributeId in (2,3,5)
group by ProductId
having count(distinct AttributeId)=3
I'm trying to select randomly few rows for each Id stored in one table where these Ids have multiple rows on this table. It's difficult to explain with words, so let me show you with an example :
Example from the table :
Id Review
1 Text11
1 Text12
1 Text13
2 Text21
3 Text31
3 Text32
4 Text41
5 Text51
6 Text61
6 Text62
6 Text63
Result expected :
Id Review
1 Text11
1 Text13
2 Text21
3 Text32
4 Text41
5 Text51
6 Text62
In fact, the table contains thousands of rows. Some Ids contain only one Review but others can contain hundreds of reviews. I would like to select 10% of these, and select at least once, all rows wich have 1-9 reviews (I saw the SELECT TOP 10 percent FROM table ORDER BY NEWID() includes the row even if it's alone)
I read some Stack topics, I think I have to use a subquery but I don't find the correct solution.
Thanks by advance.
Regards.
Try this:
DECLARE #t table(Id int, Review char(6))
INSERT #t values
(1,'Text11'),
(1,'Text12'),
(1,'Text13'),
(2,'Text21'),
(3,'Text31'),
(3,'Text32'),
(4,'Text41'),
(5,'Text51'),
(6,'Text61'),
(6,'Text62'),
(6,'Text63')
;WITH CTE AS
(
SELECT
id, Review,
row_number() over (partition by id order by newid()) rn,
count(*) over (partition by id) cnt
FROM #t
)
SELECT id, Review
FROM CTE
WHERE rn <= (cnt / 10) + 1
Result(random):
id Review
1 Text12
2 Text21
3 Text31
4 Text41
5 Text51
6 Text63