SQL Server query involving subqueries - performance issues - sql-server

I have three tables:
Table 1: | dbo.pc_a21a22 |
batchNbr Other columns...
-------- ----------------
12345
12346
12347
Table 2: | dbo.outcome |
passageId record
---------- ---------
00003 200
00003 9
00004 7
Table 3: | dbo.passage |
passageId passageTime batchNbr
---------- ------------- ---------
00001 2015.01.01 12345
00002 2016.01.01 12345
00003 2017.01.01 12345
00004 2018.01.01 12346
What I want to do: for each batchNbr in Table 1 get first its latest passageTime and the corresponding passageID from Table 3. With that passageID, get the relevant rows in Table 2 and establish whether any of these rows contains the record 200. Per passageId there are at most 2 records in Table 2
What is the most efficient way to do this?
I have already created a query that works, but it's awfully slow and thus unfit for tables with millions of rows. Any suggestion on how to either change the query or do it another way? Altering the table structure is not an option, I only have read rights to the database.
My current solution (slow):
SELECT TOP 50000
a.batchNbr,
CAST ( CASE WHEN 200 in (SELECT TOP 2 record FROM dbo.outcome where passageId in (
SELECT SubqueryResults.passageId From (SELECT Top 1 passageId FROM dbo.passage pass WHERE pass.batchNbr = a.batchNbr ORDER BY passageTime Desc) SubqueryResults
)
) then 1 else 0 end as bit) as KGT_IO_END
FROM dbo.pc_a21a22 a
The desired output is:
batchNbr 200present
--------- ----------
12345 1
12346 0

I suggest you use table joining rather than subqueries.
select
a.*, b.*
from
dbo.table1 a
join
dbo.table2 b on a.id = b.id
where
/*your where clause for filtering*/
EDIT:
You could use this as a reference Join vs. sub-query

Try this
SELECT TOP 50000 a.*, (CASE WHEN b.record = 200 THEN 1 ELSE 0 END) AS
KGT_IO_END
FROM dbo.Test1 AS a
LEFT OUTER JOIN
(SELECT record, p.batchNbr
FROM dbo.Test2 AS o
LEFT OUTER JOIN (SELECT MAX(passageId) AS passageId, batchNbr FROM
dbo.Test3 GROUP BY batchNbr) AS p ON o.passageId = p.passageId
) AS b ON a.batchNbr = b.batchNbr;
The MAX subquery is to get the latest passageId by batchNbr.
However, your example won't get the record 200, since the passageId of the record with 200 is 00001, while the latest passageId of the batchNbr 12345 is 00003.
I used LEFT OUTER JOIN since the passageId from Table2 no longer match any of the latest passageId from Table3. The resulting subquery would have no records to join to Table1. Therefore INNER JOIN would not show any records from your example data.
Output from your example data:
batchNbr KGT_IO_END
12345 0
12346 0
12347 0
Output if we change the passageId of record 200 to 00003 (the latest for 12345)
batchNbr KGT_IO_END
12345 1
12346 0
12347 0

Related

SQLite: Return NULL fields in multiple LEFT JOIN Table

I have a cumbersome query I'm building in a certain way as I'll be calling it from the C-API substituting certain values. I'm having an issue where I'm expecting NULL fields to be populating the final table.
To populate my query, I generate a date column using a recursive table expression and joining another table twice.
An example set from my table's data:
SELECT * FROM myTable;
id foreignId date value
---------- ---------- ---------- ----------
1 1 2019-12-01 100
2 1 2019-12-02 101
3 1 2019-12-03 102
4 1 2019-12-04 103
5 1 2019-12-07 104
6 2 2019-12-01 200
7 2 2019-12-02 201
8 2 2019-12-03 202
9 2 2019-12-07 203
The query I'm using:
WITH RECURSIVE dates(date) AS (
VALUES('2019-12-01')
UNION ALL
SELECT date(date, '+1 day')
FROM dates
WHERE date < '2019-12-07'
)
SELECT a.date, b.myTable, c.myTable
FROM dates a
LEFT JOIN myTable b ON a.date = b.date
LEFT JOIN myTable c ON a.date = c.date
WHERE b.foreignId = 1 AND c.foreignId = 2;
Returns the table:
date myTable myTable
---------- ---------- ----------
2019-12-01 100 200
2019-12-02 101 201
2019-12-03 102 202
2019-12-07 104 203
What I am trying to achieve:
date myTable myTable
---------- ---------- ----------
2019-12-01 100 200
2019-12-02 101 201
2019-12-03 102 202
2019-12-04 103
2019-12-05
2019-12-06
2019-12-07 104 203
I've tried using IFNULL in the SELECT statement like:
...
SELECT a.date, IFNULL(b.myTable, 0) b.myTable, IFNULL(c.myTable, 0) c.myTable
...
Which returns:
Error: near ".": syntax error
I'm not certain what the syntax error is, and haven't got that part working to test the result.
I've also tried using CROSS JOIN in place of LEFT JOIN and various combinations, but they return the same table as the LEFT JOIN. Can anyone give me some guidance, particularly anything in the documentation I may have missed?
Move the condition on the left joined tables from the where clause to the on part of the join. Otherwise, rows where the left join comes up empty are filtered out by the where clause (since the conditions are not be fulfilled): this actually turns your left join to an inner join.
WITH RECURSIVE dates(date) AS ( ...)
SELECT a.date, b.myTable, c.myTable
FROM dates a
LEFT JOIN myTable b ON a.date = b.date AND b.foreignId = 1
LEFT JOIN myTable c ON a.date = c.date AND c.foreignId = 2;

Recursive SQL Query with Defined Start Point

I'm trying to get a recursive query to work with a defined starting point.
Here's some sample data from my table called Part_v_Container:
Serial_No From_Container Part_Key Part_Operation_Key
1234 1233 5678 5
1233 1232 5678 4
1231 1230 5678 3
1230 NULL 5678 2
Basically we ship a serial number and then I need to trace it back following all the previous serial numbers From_Container through the process. In my real query I'll have the starting Serial_No be a user defined variable.
Here's my attempt:
;WITH Recursive_cte AS
(SELECT
PContainer.Serial_No
,PContainer.From_Container
,PContainer.Part_Key
,PContainer.Part_Operation_Key
FROM Part_v_Container AS PContainer
WHERE Serial_No LIKE '1234' -- Will be user defined variable
UNION ALL
SELECT
PContainer.Serial_No
,PContainer.From_Container
,PContainer.Part_Key
,PContainer.Part_Operation_Key
FROM Recursive_cte
INNER JOIN
Part_v_Container AS PContainer
ON PContainer.From_Container = Recursive_cte.Serial_No
)
SELECT *
FROM Recursive_cte
Yet this only returns one row, the Serial_No = 1234 row. My real data set has thousands of serial numbers and I need to be able to pick which ones I pick from to trace back, not a broad query that is recursive for every one in my table.
I've tried reading several articles and examples to get this to work, including the one here with no success.
Thank you in advance for your help.
You had the join fields inverted.
SQL Demo
;WITH Recursive_cte AS (
SELECT
PContainer.Serial_No
, PContainer.From_Container
, PContainer.Part_Key
, PContainer.Part_Operation_Key
FROM Part_v_Container AS PContainer
WHERE Serial_No = 1234 -- Will be user defined variable
UNION ALL
SELECT
PContainer.Serial_No
, PContainer.From_Container
, PContainer.Part_Key
, PContainer.Part_Operation_Key
FROM Recursive_cte
INNER JOIN Part_v_Container AS PContainer
ON Recursive_cte.From_Container = PContainer.Serial_No
)
SELECT *
FROM Recursive_cte
OUTPUT
| Serial_No | From_Container | Part_Key | Part_Operation_Key |
|-----------|----------------|----------|--------------------|
| 1234 | 1233 | 5678 | 5 |
| 1233 | 1232 | 5678 | 4 |

Updating 1 table from another using wheres

Trying to update one column, from another table with the highest Date.
Table 1 Example:
PartNumber | Cost
1000 | .10
1001 | .20
Table 2 Example:
PartNumber | Cost | Date
1000 | .10 | 2017-01-01
1000 | .50 | 2017-02-01
1001 | .20 | 2017-01-01
1002 | .50 | 2017-02-02
I would like to update table 1 with the most recent values from table2, which would be .50 for each... The query I use to update this has worked just fine until I realized I was not grabbing the correct Cost because there were multiples.. I now want to grab the highest dated revision.
My query:
UPDATE dex_mfgx..insp_master
SET dex_mfgx..insp_master.costperpart = t2.sct_cst_tot
FROM dex_mfgx..insp_master AS t1
INNER JOIN qad_repl..sct_det_sql AS t2
ON t1.partnum = t2.sct_part
WHERE t1.partnum = t2.sct_part and t2.sct_cst_date = MAX(t2.sct_cst_date) ;
My Error:
Msg 147, Level 15, State 1, Line 6
An aggregate may not appear in the WHERE clause unless it is in a subquery contained in a HAVING clause or a select list, and the column being aggregated is an outer reference.
Not having much luck with HAVING or GROUPING, although I havent used them much..
Any have an idea that would help?
I think I understand what you are trying to solve now. Thanks to Lamak for setting me straight as I was way off base originally.
Something like this I think is what you are looking for.
with TotalCosts as
(
SELECT t2.sct_cst_tot
, t1.partnum
, RowNum = ROW_NUMBER() over(partition by t1.partnun order by t2.sct_cst_date desc)
FROM dex_mfgx..insp_master AS t1
INNER JOIN qad_repl..sct_det_sql AS t2 ON t1.partnum = t2.sct_part
)
update t1
set costperpart = tc.sct_cst_tot
from dex_mfgx..insp_master AS t1
join TotalCosts tc on tc.partnum = t1.partnum
where tc.RowNum = 1

Joining two SQL tables based on the equality of few columns

I am trying to Create a SQL View by joining two SQL tables and return only the lowest value from second table and all the rows from first table similar to left join.
My problem can be clearly explained with the below example.
Table1
Id Product Grade Term Bid Offer
100 ABC A Q1 10 20
101 ABC A Q1 5 25
102 XYZ A Q2 25 30
103 XYZ B Q2 20 30
Table2
Id Product Grade Term TradeValue
1 ABC A Q1 100
2 ABC A Q1 95
3 XYZ B Q2 100
In the above data I want to join Table1 and Table2 when ever the columns Product,Grade and Term from both the tables are equal and return all the rows from Table1 while joining the lowest Value of the column TradeValue from Table2 to the first record of the match and making TradeValue as NULL for other rows of the resultant View and the resultant View should have the Id of Table2 as LTID
So the resultant SQL View should be
RESULT
Id Product Grade Term Bid Offer TradeValue LTID
100 ABC A Q1 10 20 95 2
101 ABC A Q1 5 25 NULL 2
102 XYZ A Q2 25 30 NULL NULL
103 XYZ B Q2 20 30 100 3
I tried using the following query
CREATE VIEW [dbo].[ViewCC]
AS
SELECT
a.Id,a.Product,a.Grade,a.Term,a.Bid,a.Offer,
b.TradeValue
FROM Table1 AS a
left JOIN (SELECT Product,Grade,Term,MIN(TradeValue) TradeValue from Table2 Group by Product,Grade,Term,) AS b
ON b.Product=a.Product
and b.Grade=a.Grade
and b.Term=a.Term
GO
The above Query returned the following data which is apt to the query I wrote but that is not what I was trying to get
Id Product Grade Term Bid Offer TradeValue
100 ABC A Q1 10 20 95
101 ABC A Q1 5 25 95 --This should be null
102 XYZ A Q2 25 30 NULL
103 XYZ B Q2 20 30 100
As we can see minimum value of TradeValue being assigned to all matching rows in Table1 and also I was not able to return Id As LTID from Table2 as I have issues with group by clause as I cannot group it by b.Id as it returns too many rows.
May I know a better way to deal with this?
You need a row number attached to each record from Table1, so that the requirement of only joining the first record from each group of Table1 can be fulfilled:
CREATE VIEW [dbo].[ViewCC]
AS
SELECT a.Id, a.Product, a.Grade, a.Term, a.Bid, a.Offer,
b.TradeValue, b.Id AS LTID
FROM (
SELECT *, ROW_NUMBER() OVER(PARTITION BY Product, Grade, Term ORDER BY Id) AS rn
FROM Table1
) a
OUTER APPLY (
SELECT TOP 1 CASE WHEN rn = 1 THEN TradeValue
ELSE NULL
END AS TradeValue, Id
FROM Table2
WHERE Product=a.Product AND Grade=a.Grade AND Term=a.Term
ORDER BY TradeValue) b
GO
OUTER APPLY returns a table expression containing either the matching record from Table2 with the lowest TradeValue, or NULL if no matching record exists.

SQL Server - Transpose rows into columns

I've searched high and low for an answer to this so apologies if it's already answered!
I have the following result from a query in SQL 2005:
ID
1234
1235
1236
1267
1278
What I want is
column1|column2|column3|column4|column5
---------------------------------------
1234 |1235 |1236 |1267 |1278
I can't quite get my head around the pivot operator but this looks like it's going to be involved. I can work with there being only 5 rows for now but a bonus would be for it to be dynamic, i.e. can scale to x rows.
EDIT:
What I'm ultimately after is assigning the values of each resulting column to variables, e.g.
DECLARE #id1 int, #id2 int, #id3 int, #id4 int, #id5 int
SELECT #id1 = column1, #id2 = column2, #id3 = column3, #id4 = column4,
#id5 = column5 FROM [transposed_table]
You also need a value field in your query for each id to aggregate on. Then you can do something like this
select [1234], [1235]
from
(
-- replace code below with your query, e.g. select id, value from table
select
id = 1234,
value = 1
union
select
id = 1235,
value = 2
) a
pivot
(
avg(value) for id in ([1234], [1235])
) as pvt
I think you'll find the answer in this answer to a slightly different question: Generate "scatter plot" result of members against sets from SQL query
The answer uses Dynamic SQL. Check out the last link in mellamokb's answer: http://www.sqlfiddle.com/#!3/c136d/14 where he creates column names from row data.
In case you have a grouped flat data structure that you want to group transpose, like such:
GRP | ID
---------------
1 | 1234
1 | 1235
1 | 1236
1 | 1267
1 | 1278
2 | 1234
2 | 1235
2 | 1267
2 | 1289
And you want its group transposition to appear like:
GRP | Column 1 | Column 2 | Column 3 | Column 4 | Column 5
-------------------------------------------------------------
1 | 1234 | 1235 | 1236 | 1267 | 1278
2 | 1234 | 1235 | NULL | 1267 | NULL
You can accomplish it with a query like this:
SELECT
Column1.ID As column1,
Column2.ID AS column2,
Column3.ID AS column3,
Column4.ID AS column4,
Column5.ID AS column5
FROM
(SELECT GRP, ID FROM FlatTable WHERE ID = 1234) AS Column1
LEFT OUTER JOIN
(SELECT GRP, ID FROM FlatTable WHERE ID = 1235) AS Column2
ON Column1.GRP = Column2.GRP
LEFT OUTER JOIN
(SELECT GRP, ID FROM FlatTable WHERE ID = 1236) AS Column3
ON Column1.GRP = Column3.GRP
LEFT OUTER JOIN
(SELECT GRP, ID FROM FlatTable WHERE ID = 1267) AS Column4
ON Column1.GRP = Column4.GRP
LEFT OUTER JOIN
(SELECT GRP, ID FROM FlatTable WHERE ID = 1278) AS Column5
ON Column1.GRP = Column5.GRP
(1) This assumes you know ahead of time which columns you will want — notice that I intentionally left out ID = 1289 from this example
(2) This basically uses a bunch of left outer joins to append 1 column at a time, thus creating the transposition. The left outer joins (rather than inner joins) allow for some columns to be null if they don't have corresponding values from the flat table, without affecting any subsequent columns.

Resources