Say I have two tables: A and B
Table A
+----+-------+
| id | value |
+----+-------+
| 1 | 20 |
| 2 | 20 |
| 3 | 10 |
| 4 | 0 |
+----+-------+
Table B
+----+-------+
| id | value |
+----+-------+
| 1 | 20 |
| 2 | 10 |
| 3 | 30 |
| 4 | 20 |
| 5 | 20 |
| 6 | 10 |
+----+-------+
If I do SELECT value, COUNT(*) AS occurrence FROM A GROUP BY value, I'll get:
+-------+------------+
| value | occurrence |
+-------+------------+
| 20 | 2 |
| 10 | 1 |
| 0 | 1 |
+-------+------------+
Based on this grouping of table A, I want to delete occurrence records from table B with the same values. In other words, I want to delete from B 2 records with value 20, 1 record with value 10, and 1 record with value 0. (Other conditions include 'do nothing if no record exists' and 'smallest id first', but I think these conditions are pretty trivial compared to the bulk of this question.)
Table B after deleting should be:
+----+-------+
| id | value |
+----+-------+
| 3 | 30 |
| 5 | 20 |
| 6 | 10 |
+----+-------+
From the official TOP documentation, doesn't seems like I can perform some JOIN to use as the TOP expression.
We could use ROW_NUMBER with CTEs here:
WITH cteA AS (
SELECT value, COUNT(*) cnt
FROM A
GROUP BY value
),
cteB AS (
SELECT *, ROW_NUMBER() OVER (PARTITION BY value ORDER BY id) rn
FROM B
)
DELETE
FROM cteB b
INNER JOIN cteA a
ON b.value = a.value
WHERE
b.rn <= a.cnt;
The logic here is that we use ROW_NUMBER to keep track of the order of each value in the B table. Then, we join to bring in the counts of each value in the A table, and we only delete B records for which the row number is strictly less than or equal to the A count.
See the demo link below to verify that the logic be correct. Note that I use a select there, not a delete, but the correct rows are being targeted for deletion.
Demo
I'm trying to get a "lineage" or similar, and also information about the first and last links (at least; all would be good), out of a table that has self-referential links between rows that have been "replaced" and rows that have replaced them. The table has a structure along these lines:
CREATE TABLE Thing (
Id INT PRIMARY KEY,
TStamp DATETIME,
Replaces INT NULL,
ReplacedBy INT NULL
);
I'm stuck with this structure. :-) It's sort of doubly-linked (yes, it's a bit silly): Each row has a unique Id, and then a row that has been "replaced" by another will have a non-NULL ReplacedBy giving the Id of the replacement row, and the replacement row will also have a link back to what it replaces in Replaces. So we can use either Replaces or ReplacedBy (or both) if we like.
Here's some sample data:
INSERT INTO Thing
(Id, TStamp, Replaces, ReplacedBy)
VALUES
(1, '2017-01-01', NULL, 11),
(2, '2017-01-02', NULL, 12),
(3, '2017-01-03', NULL, NULL),
(4, '2017-01-04', NULL, NULL),
(11, '2017-01-11', 1, NULL),
(12, '2017-01-12', 2, 22),
(22, '2017-01-22', 12, NULL);
So 1 was replaced by 11, 2 was replaced by 12, and 12 was replaced by 22.
I'd like to get the following information for each chain of links from this table in a reasonable way:
Details of the row that started the chain
Details of the final row in the chain
Details of the links in-between or at least how many links (total) there are in the chain
...filtered by a date range applied to the last row in the chain.
In an ideal universe, I'd get back something like this:
+−−−−−−−−−+−−−−−−−−+−−−−+−−−−−−−+−−−−−−−−−−−−+
| FirstId | LastId | Id | Links | TStamp |
+−−−−−−−−−+−−−−−−−−+−−−−+−−−−−−−+−−−−−−−−−−−−+
| 1 | 11 | 1 | 2 | 2017−01−01 |
| 1 | 11 | 11 | 2 | 2017−01−11 |
| 2 | 22 | 2 | 3 | 2017−01−02 |
| 2 | 22 | 12 | 3 | 2017−01−12 |
| 2 | 22 | 22 | 3 | 2017−01−22 |
+−−−−−−−−−+−−−−−−−−+−−−−+−−−−−−−+−−−−−−−−−−−−+
So far I have this query, which I could post-process to get the above:
WITH Data AS (
SELECT Id, TStamp, Replaces, ReplacedBy, 0 AS Depth
FROM Thing
UNION ALL
SELECT Thing.Id, Thing.TStamp, Thing.Replaces, Thing.ReplacedBy, Depth + 1
FROM Data
JOIN Thing
ON Thing.Replaces = Data.Id
)
SELECT *
FROM Data
WHERE ReplacedBy IS NOT NULL OR Depth > 0
ORDER BY
Id, Depth;
That gives me:
+−−−−+−−−−−−−−−−−−+−−−−−−−−−−+−−−−−−−−−−−−+−−−−−−−+
| Id | TStamp | Replaces | ReplacedBy | Depth |
+−−−−+−−−−−−−−−−−−+−−−−−−−−−−+−−−−−−−−−−−−+−−−−−−−+
| 1 | 2017−01−01 | NULL | 11 | 0 |
| 2 | 2017−01−02 | NULL | 12 | 0 |
| 11 | 2017−01−11 | 1 | NULL | 1 |
| 12 | 2017−01−12 | 2 | 12 | 0 |
| 12 | 2017−01−12 | 2 | 12 | 1 |
| 22 | 2017−01−13 | 12 | NULL | 1 |
| 22 | 2017−01−13 | 12 | NULL | 2 |
+−−−−+−−−−−−−−−−−−+−−−−−−−−−−+−−−−−−−−−−−−+−−−−−−−+
And I could use something like this to figure out (for instance) the final row of each chain:
WITH Data AS (
SELECT Id, Replaces, ReplacedBy, 0 AS Depth
FROM Thing
UNION ALL
SELECT Thing.Id, Thing.Replaces, Thing.ReplacedBy, Depth + 1
FROM Data
JOIN Thing
ON Thing.Replaces = Data.Id
),
MaxData AS (
SELECT Data.Id, Data.Depth
FROM Data
JOIN (
SELECT Id, MAX(Depth) AS MaxDepth
FROM Data
GROUP BY Id
) j ON data.Id = j.Id AND Data.Depth = j.MaxDepth
WHERE Depth > 0
)
SELECT *
FROM MaxData
ORDER BY
Id;
...which gives me:
+−−−−+−−−−−−−+
| Id | Depth |
+−−−−+−−−−−−−+
| 11 | 1 |
| 12 | 1 |
| 22 | 2 |
+−−−−+−−−−−−−+
...but I've lost the starting point and the points along the way.
I have the strong feeling I'm missing something really straight-forward — but clever — that would let me get this largely with the query rather than post-processing, some kind of join with a "min" and "max" query (but not like my one above). What would it be?
The table doesn't have any indexes on Replaces or ReplacedBy, but we could add any needed. The table is only lightly used (roughly 300k rows and probably only a couple of hundred updates/inserts a day).
I'm limited to SQL Server 2008 features.
Inspired by Gordon Linoff's answer and HABO's comment which highlighted something Gordon was doing that was critical, I:
Removed the SQL Server 2012+ FIRST_VALUE function, replacing it with a CROSS JOIN on an "overview" query of the data
Included the Links count in the overview query
Removed the reliance on t in Gordon's WHERE NOT EXISTS (SELECT 1 FROM Thing t2 WHERE t2.ReplacedBy = t.id), which (at last on SQL Server 2008) wasn't bound to anything
Filtered out rows that weren't replaced
Below, I also add the date filtering mentioned in the question
...filtered by a date range applied to the last row in the chain.
...which Gordon didn't cover at all, and changes our approach, but only in terms of the arrow of time.
So, first, without the date criteria, sticking fairly close to Gordon's answer:
WITH Data AS (
SELECT Id AS FirstId, Id, TStamp, Replaces, ReplacedBy, 0 AS Depth
FROM Thing
WHERE Replaces IS NULL AND ReplacedBy IS NOT NULL
UNION ALL
SELECT d.FirstId, t.Id, t.TStamp, t.Replaces, t.ReplacedBy, d.Depth + 1
FROM Data d
JOIN Thing t ON t.Replaces = d.Id
),
Overview AS (
SELECT FirstId, MAX(Id) AS LastId, COUNT(*) AS Links
FROM Data
GROUP BY
FirstId
)
SELECT d.FirstId, o.LastId, d.Id, o.Links, d.Depth, d.TStamp
FROM Data d
CROSS APPLY (
SELECT LastId, Links
FROM Overview
WHERE FirstId = d.FirstId
) o
ORDER BY
d.FirstId, d.Depth
;
The critical parts of that are grabbing the seed Id as FirstId here:
SELECT Id AS FirstId, Id, TStamp, Replaces, ReplacedBy, 0 AS Depth
FROM Thing
WHERE Replaces IS NULL AND ReplacedBy IS NOT NULL
and then propagating it through the results of the recursive join:
SELECT d.FirstId, t.Id, t.TStamp, t.Replaces, t.ReplacedBy, d.Depth + 1
FROM Data d
JOIN Thing t ON t.Replaces = d.Id
Just adding that to my original query gives us most of what I wanted. Then we add a second query to get the LastId for each FirstId (Gordon did it as a FIRST_VALUE over a partition, but I can't do that in SQL Server 2008) and using an overview query also lets me grab the number of links. We cross-apply that on the basis of the FirstId value to get the overall results I wanted.
The query above returns the following for the sample data:
+−−−−−−−−−+−−−−−−−−+−−−−+−−−−−−−+−−−−−−−+−−−−−−−−−−−−+
| FirstId | LastId | Id | Links | Depth | TStamp |
+−−−−−−−−−+−−−−−−−−+−−−−+−−−−−−−+−−−−−−−+−−−−−−−−−−−−+
| 1 | 11 | 1 | 2 | 0 | 2017-01-01 |
| 1 | 11 | 11 | 2 | 1 | 2017-01-11 |
| 2 | 22 | 2 | 3 | 0 | 2017-01-02 |
| 2 | 22 | 12 | 3 | 1 | 2017-01-12 |
| 2 | 22 | 22 | 3 | 2 | 2017-01-13 |
+−−−−−−−−−+−−−−−−−−+−−−−+−−−−−−−+−−−−−−−+−−−−−−−−−−−−+
...e.g., exactly what I wanted, plus Depth if I want (so I know what order the intermediary links were in).
If we wanted to include rows that were never replaced, we'd just change
WHERE Replaces IS NULL AND ReplacedBy IS NOT NULL
to
WHERE Replaces IS NULL
Giving us:
+−−−−−−−−−+−−−−−−−−+−−−−+−−−−−−−+−−−−−−−+−−−−−−−−−−−−+
| FirstId | LastId | Id | Links | Depth | TStamp |
+−−−−−−−−−+−−−−−−−−+−−−−+−−−−−−−+−−−−−−−+−−−−−−−−−−−−+
| 1 | 11 | 1 | 2 | 0 | 2017-01-01 |
| 1 | 11 | 11 | 2 | 1 | 2017-01-11 |
| 2 | 22 | 2 | 3 | 0 | 2017-01-02 |
| 2 | 22 | 12 | 3 | 1 | 2017-01-12 |
| 2 | 22 | 22 | 3 | 2 | 2017-01-13 |
| 3 | 3 | 3 | 1 | 0 | 2017-01-03 |
| 4 | 4 | 4 | 1 | 0 | 2017-01-04 |
+−−−−−−−−−+−−−−−−−−+−−−−+−−−−−−−+−−−−−−−+−−−−−−−−−−−−+
But we've ignored the date criteria required by the question:
...filtered by a date range applied to the last row in the chain.
To do that without building a massive temporary result set, we have to work backward: Instead of selecting the starting point (the first entry in a chain, Replaces IS NULL), we need to select the ending point (the last entry in a chain, ReplacedBy IS NULL), and then invert our logic working back through the chain. It's largely a matter of:
Swapping FirstId with LastId
Swapping Replaces with ReplacedBy (convenient the table had both!)
Using MIN to get the first ID in the chain rather than MAX to get the last
Using d.Depth - 1 rather than d.Depth + 1
Then fixing-up Depth based on Links once we know it in our final select, to get those nice values where 0 = first link rather than some varying negative number: o.Links + d.Depth - 1 AS Depth
All of which gives us:
WITH Data AS (
SELECT Id AS LastId, Id, TStamp, Replaces, ReplacedBy, 0 AS Depth
FROM Thing
WHERE ReplacedBy IS NULL AND Replaces IS NOT NULL
-- Filtering by date of last entry would go here
UNION ALL
SELECT d.LastId, t.Id, t.TStamp, t.Replaces, t.ReplacedBy, d.Depth - 1
FROM Data d
JOIN Thing t ON t.ReplacedBy = d.Id
),
Overview AS (
SELECT LastId, MIN(Id) AS FirstId, COUNT(*) AS Links
FROM Data
GROUP BY
LastId
)
SELECT o.FirstId, d.LastId, d.Id, o.Links, o.Links + d.Depth - 1 AS Depth, d.TStamp
FROM Data d
CROSS APPLY (
SELECT FirstId, Links
FROM Overview
WHERE LastId = d.LastId
) o
ORDER BY
o.FirstId, d.Depth
;
So for instance, if we used
AND TStamp BETWEEN '2017-01-12' AND '2017-02-01'
where I have
-- Filtering by date of last entry would go here
above, with our sample data we'd get this result:
+−−−−−−−−−+−−−−−−−−+−−−−+−−−−−−−+−−−−−−−+−−−−−−−−−−−−+
| FirstId | LastId | Id | Links | Depth | TStamp |
+−−−−−−−−−+−−−−−−−−+−−−−+−−−−−−−+−−−−−−−+−−−−−−−−−−−−+
| 2 | 22 | 2 | 3 | 0 | 2017−01−02 |
| 2 | 22 | 12 | 3 | 1 | 2017−01−12 |
| 2 | 22 | 22 | 3 | 2 | 2017−01−13 |
+−−−−−−−−−+−−−−−−−−+−−−−+−−−−−−−+−−−−−−−+−−−−−−−−−−−−+
...because the last link the Id = 1 chain is outside the date range, so we don't include it.
This is a little tricky. Arrange the CTE to start at the beginning of each list. That makes the subsequent processing easier:
WITH Data AS (
SELECT Id as FirstId, Id, TStamp, Replaces, ReplacedBy, 0 AS Depth
FROM Thing t
WHERE NOT EXISTS (SELECT 1 FROM Thing t2 WHERE t2.ReplacedBy = t.id)
UNION ALL
SELECT d.FirstId, t.Id, t.TStamp, t.Replaces, t.ReplacedBy, d.Depth + 1
FROM Data d JOIN
Thing t
ON t.Replaces = d.Id
)
SELECT d.*,
FIRST_VALUE(id) OVER (PARTITION BY FirstId ORDER BY Depth DESC) as LastId
FROM Data d;
Then, you can use FIRST_VALUE() with a reverse sort to get the last value in the chain.
This returns chains that have no links. You can add a filter to remove these.
I have 5 columns in SQL that I need to turn into a cross tab in Crystal.
This is what I have:
Key | RELATIONSHIP | DISABLED | LIMITED | RURAL | IMMIGRANT
-----------------------------------------------------------------
1 | Other Dependent | Yes | No | No | No
2 | Victim/Survivor | No | No | No | No
3 | Victim/Survivor | Yes | No | No | No
4 | Child | No | No | No | No
5 | Victim/Survivor | No | No | No | No
6 | Victim/Survivor | No | No | No | No
7 | Child | No | No | No | No
8 | Victim/Survivor | No | Yes | Yes | Yes
9 | Child | No | Yes | Yes | Yes
10 | Child | No | Yes | Yes | Yes
This is what I want the cross tab to look like (Distinct count on Key):
| Victim/Survivor | Child | Other Dependent | Total |
--------------------------------------------------------------
| DISABLED | 1 | 0 | 1 | 2 |
--------------------------------------------------------------
| LIMITED | 1 | 2 | 0 | 3 |
--------------------------------------------------------------
| RURAL | 1 | 2 | 0 | 3 |
--------------------------------------------------------------
| IMMIGRANT | 1 | 2 | 0 | 3 |
--------------------------------------------------------------
| TOTAL | 4 | 6 | 1 | 11 |
--------------------------------------------------------------
I used this formula in Crystal in an effort to combine 4 columns (Field name = {#OTHERDEMO})...
IF {TABLE.DISABLED} = "YES" THEN "DISABLED" ELSE
IF {TABLE.LIMITED} = "YES" THEN "LIMITED" ELSE
IF {TABLE.IMMIGRANT} = "YES" THEN "IMMIGRANT" ELSE
IF {TABLE.RURAL} = "YES" THEN "RURAL"
...then made the cross-tab with #OTHERDEMO as the rows, RELATIONSHIP as the Columns with a distinct count on KEY:
Problem is, once crystal hits the first "Yes" it stops counting thus not categorizing correctly in the cross-tab. So I get a table that counts the DISABILITY first and gives the correct display, then counts the Limited and gives some info, but then dumps everything else.
In the past, I have done mutiple conditional formulas...
IF {TABLE.DISABLED} = "YES" AND {TABLE.RELATIONSHIP} = "Victim/Survivor" THEN {TABLE.KEY} ELSE {#NULL}
(the #null formula is because Crystal, notoriously, gets confused with nulls.)
...then did a distinct count on Key, and finally summed it in the footer.
I am convinced there is another way to do this. Any help/ideas would be greatly appreciated.
If you unpivot those "DEMO" columns into rows it will make the crosstab super easy...
select
u.[Key],
u.[RELATIONSHIP],
u.[DEMO]
from
Table1
unpivot (
[b] for [DEMO] in ([DISABLED], [LIMITED], [RURAL], [IMMIGRANT])
) u
where
u.[b] = 'Yes'
SqlFiddle
or if you were stuck on SQL2000 compatibility level you could manually unpivot the Yes values...
select [Key], [REALTIONSHIP], [DEMO] = cast('DISABLED' as varchar(20))
from Table1
where [DISABLED] = 'Yes'
union
select [Key], [REALTIONSHIP], [DEMO] = cast('LIMITED' as varchar(20))
from Table1
where [LIMITED] = 'Yes'
union
select [Key], [REALTIONSHIP], [DEMO] = cast('RURAL' as varchar(20))
from Table1
where [RURAL] = 'Yes'
union
select [Key], [REALTIONSHIP], [DEMO] = cast('IMMIGRANT' as varchar(20))
from Table1
where [IMMIGRANT] = 'Yes'
For the crosstab, use a count on the Key column (aka row count), [DEMO] on rows, and [RELATIONSHIP] on columns.
Hi I have doubt in sql server
Trantable:
empid | deptid | Projectname | Transactionid
1 |10 | test | 1
2 |11 | test1 | 2
2 |10 | jai | 3
2nd table: dimemp ....> here dimemp is scdtype2 dimension.its all ready done
empkey | empid | ename | flag
1 | 1 | a | 1
2 | 2 | b | 1
3 | -1 | na | 1
3rd table: dimdept------>here dimdept is scdtype2 dimension.implementaion allready done.
deptkey | deptid | deptname | flag
1 | 10 | hr | 1
2 | 11 | ceo | 1
3 | -1 | NA | 1
Here I want load trantable data into facttran table with corresponding keys. here transactionid is unique column
to identiy unique record.
Facttran table structure look like below and factran we need to maintain scd type1 data.
empkey | deptkey | projectname |transactionid
I tried like below query
merge into facttran target
using (select ISNULL(a.empkey, (select empkey from Dimemp where empid = -1)) empkey,ISNULL(b.deptkey, (select deptkey from dimdept where deptid = -1)) deptkey, c.projectname, c.transactionid
from trantable c
left join dimemp a on a.empid=c.empid and a.flag=1
left join dimdept b on b.deptid=c.deptid and b.flag=1)source
on target.transactionid=source.transactionid
when not matched
then insert ([deptkey],[empkey],[projectname],[transactionid])
values(source.deptkey,source.empkey,source.projectname,source.transactionid)
when matched
then update set target.empkey=source.empkey ,
target.deptkey=source.deptkey,
target.projectname=source.projectname,
target.transactionid=source.transactionid ;
then I got output like below
Table :facttran
empkey | deptkey | projectname |transactionid
1 | 1 | test | 1
2 | 2 | test1 | 2
2 | 1 | Jai | 3
upto now its working fine.
2nd day in my trantable few records updated and few records insert in sourc trantable.based on below table data I want update in facttable with corresponding key.
2nd table: dimemp ....> here dimemp is scdtype2 dimension
empkey | empid | ename | flag
1 | 10 | a | 0
2 | 11 | b | 1
3 | -1 | na | 1
4 | 10 | aaa | 1
3rd table: dimdept------>here dimdept is scdtype2 dimension.implementaion allready done.
deptkey | deptid | deptname | flag
1 | 10 | hr | 0
2 | 20 | ceo | 1
3 | -1 | NA | 1
4 | 10 |hrdept | 1
Trantable:
empid | deptid | Projectname | Transactionid
1 |11 | test | 1 ------record updated in source side here deptid changed from 10 to 11
1 |11 | test123 | 2 -------Here empid changed from empid 2 to 1 and projectname changed test1 to test123
2 |10 | jai | 3 ------here no records are not changed
1 |10 | cod | 4 ----------new rocrd is came
based on above trantable.I want facttran table data look like below.
Table :facttran
empkey | deptkey | projectname |transactionid
1 | 2 | test | 1
1 | 2 | test123 | 2
2 | 1 | Jai | 3
4 | 10 | cod | 4
when I ran 2nd time with same query.I am not able get to expected result.
here mainily source trantable related transactioni id is exist or not in facttran table .if not exist then we need to insert correspondig dimensionkeys with lates flaf=1
values.if we found transactionid exist in fact table then we need to updated existing dimension corresonding key.
suppose if we take transactionid=1 records here only chnaged deptid not empid that time we donot need update empid corresponding lates flag=1 corresondingkey
we need check exisig transaction id is updated each dimension need to check exist or not if not exist latest flag=1 related corresponding key.
if exist we donot need to updated that one.if new reocrds came then we need to insert with latest flag=1 corresponding keys in factran table.
please tell me how to write query to achive this task in sql server.
I work with SQL Server and I need to merge multiples rows only if they have the same value between them in two specific columns (Col_01 and Col_02 in my example).
When I merge them I have to sum some columns (Col_03 and Col_04 in my example).
I think an example will be the more explicit. Here is my table simplified :
| ID | Col_01 | Col_02 | Col_03 | Col_04 |
| 1 | ABC | DEF | 2 | 2 |
| 2 | ABC | DEF | 1 | 0 |
| 3 | DEF | GHI | 0 | 2 |
| 4 | ABC | GHI | 1 | 0 |
| 5 | JKL | GHI | 0 | 2 |
And here is what I want after my update :
| ID | Col_01 | Col_02 | Col_03 | Col_04 |
| 2 | ABC | DEF | 3 | 2 |
| 3 | DEF | GHI | 0 | 2 |
| 4 | ABC | GHI | 1 | 0 |
| 5 | JKL | GHI | 0 | 2 |
I merged ID 1 and ID 2 because they had the same Col_01 and the same Col_02.
I tried a query like that
SELECT MAX(ID), Col_01, Col_02, SUM(Col_03), SUM(Col_04)
FROM Table
GROUP BY Col_01, Col_02
I've got what the rows merged but I loose the not merged ones.
I don't know how to properly use it in an UPDATE query in order to merge the rows with the same (Col_01, Col_02) and keep the others. Can you help me to do this ?
So you actually need to delete some rows from original table. Use MERGE statement for that:
MERGE Table tgt
USING (SELECT MAX(ID) as ID, Col_01, Col_02, SUM(Col_03) AS Col_03, SUM(Col_04) AS Col_04
FROM Table
GROUP BY Col_01, Col_02) src ON tgt.ID = srs.ID
WHEN MATCHED THEN
UPDATE
SET Col_03 = src.Col_03, Col_04 = src.Col_04
WHEN NOT MATCHED BY SOURCE THEN
DELETE
Try this:
UPDATE T SET
T.Col_03 = G.Col_03,
T.Col_04 = G.Col_04
FROM Table AS T INNER JOIN
(SELECT MAX(ID) AS ID, Col_01, Col_02, SUM(Col_03) AS Col_03, SUM(Col_04) AS Col_04
FROM Table
GROUP BY Col_01, Col_02) AS G
ON G.ID = T.ID