Trying to optimize large sql. Have included explain output [closed] - query-optimization

Question

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 8 years ago.
Improve this question
Below is the mysql select statement followed by the explain output.
I have tried adding indexes where I thought was appropriate but with no extra speed.
There are 2 rows in the 'explain' output that stand out (id 7 and 8 where the rows are over 35,000).
I think the sql needs to be restructured but I don't know how to change it. Any help would be appreciated.
EXPLAIN SELECT wo.JobStatusID
,wo.JobNo
,ql.RackCode
,ql.Description
,c.CustName
,s.ShipDate
,s.Qty
,pl.PrepLocation
,wl.WeldLocation
,line.WeldLine
,pm.Initials
,pqty.ShipTotal
,pqty.POTotal
,po.POCount
,s2.Notes
,bm.UnitQty
,labour.AssemblyLabour
,labour.WeldLabour
,labour.TotalGUTLabour + labour.TotalGUTSetUp / pqty.POTotal AS GUTLabour
,labour.TotalPrepLabour + labour.TotalPrepSetUp / pqty.POTotal AS PrepLabour
,pqty.POTotal
FROM WorkOrders wo
JOIN (
SELECT JobNo
,MasterJobNo AS RelevantJobNo
FROM WorkOrders
WHERE MasterJobNo != 0
UNION
SELECT JobNo
,JobNo AS RelevantJobNo
FROM WorkOrders
WHERE MasterJobNo = 0
) AS r
ON r.JobNo = wo.JobNo
LEFT JOIN QuoteLog ql
ON ql.QuoteID = wo.QuoteID
AND wo.FOCPlayArea = 0
LEFT JOIN Customer c
ON c.CustID = ql.CustID
LEFT JOIN WOSchedule s
ON s.JobNo = wo.JobNo
AND s.ObsoleteInd = 0
LEFT JOIN (
SELECT JobNo
,Notes
FROM WOSchedule
ORDER BY ShipDate DESC
) AS s2
ON s2.JobNo = wo.JobNo
LEFT JOIN PrepLocation pl
ON pl.PrepLocationID = wo.PrepLocationID
LEFT JOIN WeldLocation wl
ON wl.WeldLocationID = wo.WeldLocationID
LEFT JOIN ProgramManager pm
ON pm.ProgramManagerID = wo.ProgramManagerID
LEFT JOIN (
SELECT JobNo
,SUM(POQty) AS POTotal
,SUM(ShipQty) AS ShipTotal
FROM WOOrderDetails wod
JOIN WODetailType wodt
ON wodt.DetailTypeID = wod.DetailTypeID
AND wodt.ReleaseModeID = 1
AND wodt.Inactive = 0
WHERE wod.ObsoleteInd = 0
GROUP BY wod.JobNo
) AS pqty
ON pqty.JobNo = wo.JobNo
LEFT JOIN (
SELECT Jobno
,COUNT(*) AS POCount
FROM WOPOs
GROUP BY JobNo
) AS po
ON po.JobNo = wo.JobNo
LEFT JOIN BOMMaterialList bm
ON bm.JobNo = r.RelevantJobNo
LEFT JOIN WeldLine line
ON wo.WeldLineID = line.WeldLineID
LEFT JOIN (
SELECT x.JobNo
,x.BOMDetailID
,SUM(x.TotalPrepLabour) AS TotalPrepLabour
,SUM(x.TotalPrepSetUp) AS TotalPrepSetUp
,SUM(x.TotalGUTLabour) AS TotalGUTLabour
,SUM(x.TotalGUTSetUp) AS TotalGUTSetUp
,SUM(x.WeldLabour * x.WeldEfficiency) AS WeldLabour
,SUM(AssemblyLabour * AssemblyEfficiency * UnitQty) AS AssemblyLabour
FROM (
SELECT bm.JobNo
,bm.BOMDetailID
,bm.UnitQty
,SUM(bm.UnitQty * pl.HitTime * pl.NoHits * pl.NoMen) AS TotalPrepLabour
,SUM(pl.SetUp) AS TotalPrepSetUp
,SUM(bm.UnitQty * gl.HitTime * gl.NoHits * gl.NoMen) AS TotalGUTLabour
,SUM(gl.SetUp) AS TotalGUTSetUp
,bm.WeldLabour
,bm.WeldEfficiency
,bm.AssemblyLabour
,bm.AssemblyEfficiency
FROM BOMMaterialList bm
LEFT JOIN BOMPrepLabour pl
ON pl.BOMDetailID = bm.BOMDetailID
LEFT JOIN BOMGUTLabour gl
ON gl.BOMDetailID = bm.BOMDetailID
GROUP BY BOMDetailID
) AS x
GROUP BY x.JobNo
) AS labour
ON bm.BOMDetailID = labour.BOMDetailID
WHERE ( wo.JobStatusID = 1
OR wo.JobStatusID = 2
OR wo.JobStatusID = 3
)
GROUP BY wo.JobNo
ORDER BY wo.JobStatusID
,wl.WeldLocation
,line.WeldLine
,wo.JobNo;
id select_type table type possible_keys key key_len ref rows Extra
1 PRIMARY <derived2> ALL NULL NULL NULL NULL 1418 "Using temporary; Using filesort"
1 PRIMARY wo eq_ref PRIMARY,IDX_1 PRIMARY 4 r.JobNo 1 "Using where"
1 PRIMARY ql eq_ref PRIMARY PRIMARY 4 sbpdf3_custom014.wo.QuoteID 1
1 PRIMARY c eq_ref PRIMARY PRIMARY 2 sbpdf3_custom014.ql.CustID 1
1 PRIMARY s ref PRIMARY,IDX_2 PRIMARY 4 r.JobNo 28
1 PRIMARY <derived4> ALL NULL NULL NULL NULL 2884
1 PRIMARY pl eq_ref PRIMARY PRIMARY 4 sbpdf3_custom014.wo.PrepLocationID 1
1 PRIMARY wl eq_ref PRIMARY PRIMARY 4 sbpdf3_custom014.wo.WeldLocationID 1
1 PRIMARY pm eq_ref PRIMARY PRIMARY 4 sbpdf3_custom014.wo.ProgramManagerID 1
1 PRIMARY <derived5> ALL NULL NULL NULL NULL 1244
1 PRIMARY <derived6> ALL NULL NULL NULL NULL 1310
1 PRIMARY bm ref IDX1 IDX1 4 r.RelevantJobNo 19
1 PRIMARY line ref PRIMARY PRIMARY 4 sbpdf3_custom014.wo.WeldLineID 1
1 PRIMARY <derived7> ALL NULL NULL NULL NULL 1102
7 DERIVED <derived8> ALL NULL NULL NULL NULL 35736 "Using temporary; Using filesort"
8 DERIVED bm index NULL PRIMARY 4 NULL 35268
8 DERIVED pl ref PRIMARY PRIMARY 4 sbpdf3_custom014.bm.BOMDetailID 1
8 DERIVED gl ref PRIMARY PRIMARY 4 sbpdf3_custom014.bm.BOMDetailID 1
6 DERIVED WOPOs index NULL IDX_1 4 NULL 1804 "Using index"
5 DERIVED wod ref IDX_1 IDX_1 1 1660 "Using where; Using temporary; Using filesort"
5 DERIVED wodt eq_ref PRIMARY PRIMARY 2 sbpdf3_custom014.wod.DetailTypeID 1 "Using where"
4 DERIVED WOSchedule ALL NULL NULL NULL NULL 2884 "Using filesort"
2 DERIVED WorkOrders range IDX_2 IDX_2 4 NULL 15 "Using where; Using index"
3 UNION WorkOrders ref IDX_2 IDX_2 4 689 "Using index"
NULL "UNION RESULT" <union2,3> ALL NULL NULL NULL NULL NULL

Answer 1

There's a lot to look through here, but I did spot this subquery:
SELECT JobNo
,MasterJobNo AS RelevantJobNo
FROM WorkOrders
WHERE MasterJobNo != 0
UNION
SELECT JobNo
,JobNo AS RelevantJobNo
FROM WorkOrders
WHERE MasterJobNo = 0
which I believe could be re-written like so:
SELECT JobNo
,CASE WHEN MasterJobNo != 0
THEN MasterJobNo
ELSE JobNo END AS RelevantJobNo
FROM WorkOrders
And once we re-write it this way, we wonder why it needs to be a subquery, rather than simply joining to the table. And then I look again and see that we've already used this table in the query, and so we can eliminate that join entirely, and just include the CASE expression in the appropriate places. Moreover, this expression is only used in one place that I could spot (the join to BOMMaterialList).
I also noticed this subquery:
SELECT JobNo
,Notes
FROM WOSchedule
ORDER BY ShipDate DESC
I think we can eliminate this, as well. ORDER BY without a LIMIT has no meaning here when used as a derived table, and again: this table is already included in the query in another place. The only difference between the data provided this s2 subquery and the prior s table is the check on the ObsoleteInd field, and the only place data from s2 is used elsewhere is the Notes field in the select clause. The s table is only used in select list of the ShipDate and Qty items.
So I can get us down to this, which should show some improvement:
SELECT wo.JobStatusID
,wo.JobNo
,ql.RackCode
,ql.Description
,c.CustName
,case when s.ObsoleteInd = 0 then s.ShipDate else null end as ShipDate
,case when s.ObsoleteInd = 0 then s.Qty else null end as Qty
,pl.PrepLocation
,wl.WeldLocation
,line.WeldLine
,pm.Initials
,pqty.ShipTotal
,pqty.POTotal
,po.POCount
,s.Notes
,bm.UnitQty
,labour.AssemblyLabour
,labour.WeldLabour
,labour.TotalGUTLabour + labour.TotalGUTSetUp / pqty.POTotal AS GUTLabour
,labour.TotalPrepLabour + labour.TotalPrepSetUp / pqty.POTotal AS PrepLabour
,pqty.POTotal
FROM WorkOrders wo
LEFT JOIN QuoteLog ql
ON ql.QuoteID = wo.QuoteID
AND wo.FOCPlayArea = 0
LEFT JOIN Customer c
ON c.CustID = ql.CustID
LEFT JOIN WOSchedule s
ON s.JobNo = wo.JobNo
LEFT JOIN PrepLocation pl
ON pl.PrepLocationID = wo.PrepLocationID
LEFT JOIN WeldLocation wl
ON wl.WeldLocationID = wo.WeldLocationID
LEFT JOIN ProgramManager pm
ON pm.ProgramManagerID = wo.ProgramManagerID
LEFT JOIN (
SELECT JobNo
,SUM(POQty) AS POTotal
,SUM(ShipQty) AS ShipTotal
FROM WOOrderDetails wod
JOIN WODetailType wodt
ON wodt.DetailTypeID = wod.DetailTypeID
AND wodt.ReleaseModeID = 1
AND wodt.Inactive = 0
WHERE wod.ObsoleteInd = 0
GROUP BY wod.JobNo
) AS pqty
ON pqty.JobNo = wo.JobNo
LEFT JOIN (
SELECT Jobno
,COUNT(*) AS POCount
FROM WOPOs
GROUP BY JobNo
) AS po
ON po.JobNo = wo.JobNo
LEFT JOIN BOMMaterialList bm
ON bm.JobNo = CASE WHEN wo.MasterJobNo != 0 THEN wo.MasterJobNo ELSE wo.JobNo END
LEFT JOIN WeldLine line
ON wo.WeldLineID = line.WeldLineID
LEFT JOIN (
SELECT x.JobNo
,x.BOMDetailID
,SUM(x.TotalPrepLabour) AS TotalPrepLabour
,SUM(x.TotalPrepSetUp) AS TotalPrepSetUp
,SUM(x.TotalGUTLabour) AS TotalGUTLabour
,SUM(x.TotalGUTSetUp) AS TotalGUTSetUp
,SUM(x.WeldLabour * x.WeldEfficiency) AS WeldLabour
,SUM(AssemblyLabour * AssemblyEfficiency * UnitQty) AS AssemblyLabour
FROM (
SELECT bm.JobNo
,bm.BOMDetailID
,bm.UnitQty
,SUM(bm.UnitQty * pl.HitTime * pl.NoHits * pl.NoMen) AS TotalPrepLabour
,SUM(pl.SetUp) AS TotalPrepSetUp
,SUM(bm.UnitQty * gl.HitTime * gl.NoHits * gl.NoMen) AS TotalGUTLabour
,SUM(gl.SetUp) AS TotalGUTSetUp
,bm.WeldLabour
,bm.WeldEfficiency
,bm.AssemblyLabour
,bm.AssemblyEfficiency
FROM BOMMaterialList bm
LEFT JOIN BOMPrepLabour pl
ON pl.BOMDetailID = bm.BOMDetailID
LEFT JOIN BOMGUTLabour gl
ON gl.BOMDetailID = bm.BOMDetailID
GROUP BY BOMDetailID
) AS x
GROUP BY x.JobNo
) AS labour
ON bm.BOMDetailID = labour.BOMDetailID
WHERE ( wo.JobStatusID = 1
OR wo.JobStatusID = 2
OR wo.JobStatusID = 3
)
GROUP BY wo.JobNo
ORDER BY wo.JobStatusID
,wl.WeldLocation
,line.WeldLine
,wo.JobNo;
There are likely a few other improvements like this possible, but I suspect most improvement will come from examining your index use.