SQL Server - using Over/Partition By in complex query - sql-server

Question

We have many complex queries which involve a lot of columns and joins (see the example below) that are implemented as views.
In some cases these queries return duplicate rows which then have to be programmatically removed by the consuming app. Therefore, we would like to enhance the SQL query to eliminate the duplicates, and speed up the retrieval process.
I know that I can use OVER / PARTITION BY logic to do this, but I am not sure of how to modify the queries to obtain a working syntax.
Here is an example:
SELECT
Main.MfgOrder.OrderNumber,
Main.MfgOrder.DesignBOMID,
Main.Design_Plant.PlantID,
Main.MfgOrder_Operation.OrderOpID,
Main.MfgOrder_Operation.DesignOpID,
Main.MfgOrder_Operation.OpSeq,
Main.MfgOrder_Operation.Description,
Main.MfgOrder_Operation.CompletionStatus,
Main.MfgOrder__Shift.OrderShiftID,
Main.MfgOrder__Shift.WorkCenterMachineID,
Main.MfgOrder___Event.OrderEventID,
Main.MfgOrder____Reel.OrderReelID,
Main.MfgOrder____Reel.ReelNumber,
Main.MfgOrder____Reel.Location,
Main.MfgOrder____Reel.Test_Status AS Test_Status_Reel,
Main.MfgOrder____Reel.Test_Disposition AS Test_Disposition_Reel,
Main.MfgOrder____Reel.LabReleased,
Main.MfgOrder____Reel.ShipReelsBypassSet,
Main.MfgOrder_____Length.OrderLengthID,
Main.MfgOrder_____Length.LengthType,
Main.MfgOrder_____Length.LocationOnReel,
Main.MfgOrder_____Length.LocationOnLength,
Main.MfgOrder_____Length.TrialNumber,
Main.MfgOrder_____Length.SampleNumber,
Main.MfgOrder_____Length.PrintNumber,
Main.MfgOrder_____Length.Test_Status AS Test_Status_Length,
Main.MfgOrder_____Length.Test_Category,
Main.MfgOrder_____Length.Test_Disposition AS Test_Disposition_Length,
Main.MfgOrder_____Length.SampleSubmittedBy,
Main.MfgOrder_____Length.SampleSubmittedDate,
Main.MfgOrder_____Length.BypassTesting,
Main.MfgOrder_____Length_OperatorQty.Sample1Destination,
Main.MfgOrder_____Length_OperatorQty.Sample2Destination,
Main.MfgOrder_____Length_OperatorQty.Sample3Destination,
Main.MfgOrder______Component.OrderComponentID,
Main.MfgOrder______Component.DesignComponentID,
Main.MfgOrder______Component.ItemNo,
Main.MfgOrder_______Test.LabTestID,
Main.MfgOrder_______Test.OrderTestID,
Main.MfgOrder_______Test.TestComplete,
Main.MfgOrder_______Test.TestStatus,
Main.MfgOrder________Marker2.OrderMarkerID,
Master.Color.ColorName,
Master.LabTest.ExcludeFromPassFail,
CASE
WHEN Main.Design_Component.Component_Label IS NULL
THEN 'Unknown'
ELSE Main.Design_Component.Component_Label
END AS Component_Label
FROM
Main.MfgOrder
INNER JOIN
Main.Design__BOM ON Main.MfgOrder.DesignBOMID = Main.Design__BOM.DesignBOMID
INNER JOIN
Main.Design_Plant ON Main.Design__BOM.DesignPlantID = Main.Design_Plant.DesignPlantID
INNER JOIN
Main.MfgOrder_Operation ON Main.MfgOrder.OrderNumber = Main.MfgOrder_Operation.OrderNumber
INNER JOIN
Main.MfgOrder__Shift ON Main.MfgOrder_Operation.OrderOpID = Main.MfgOrder__Shift.OrderOpID
INNER JOIN
Main.MfgOrder___Event ON Main.MfgOrder__Shift.OrderShiftID = Main.MfgOrder___Event.OrderShiftID
INNER JOIN
Main.MfgOrder____Reel ON Main.MfgOrder___Event.OrderEventID = Main.MfgOrder____Reel.OrderEventID
INNER JOIN
Main.MfgOrder_____Length ON Main.MfgOrder____Reel.OrderReelID = Main.MfgOrder_____Length.OrderReelID
LEFT OUTER JOIN
Main.MfgOrder______Component ON Main.MfgOrder_____Length.OrderLengthID = Main.MfgOrder______Component.OrderLengthID
LEFT OUTER JOIN
Main.MfgOrder_______Test ON Main.MfgOrder______Component.OrderComponentID = Main.MfgOrder_______Test.OrderComponentID
LEFT OUTER JOIN
Main.MfgOrder________Marker2 ON Main.MfgOrder_______Test.OrderTestID = Main.MfgOrder________Marker2.OrderTestID
LEFT OUTER JOIN
Main.Design_Component ON Main.MfgOrder______Component.DesignComponentID = Main.Design_Component.DesignComponentID
LEFT OUTER JOIN
Master.Color ON Main.MfgOrder______Component.TapeColorID = Master.Color.ColorNumber
LEFT OUTER JOIN
Master.LabTest ON Main.MfgOrder_______Test.LabTestID = Master.LabTest.LabTestID
LEFT OUTER JOIN
Main.MfgOrder_____Length_OperatorQty ON Main.MfgOrder______Component.OrderLengthID = Main.MfgOrder_____Length_OperatorQty.OrderLengthID

Answer 1

you can use row_number as below: Below query will not select duplicate only on OrderNumber, if you need to add other columns you add accordingly
Select * from (
Select
RowN = Row_Number() over( partition by Main.MfgOrder.OrderNumber order by Main.MfgOrder.OrderNumber),
--- All your select columns and all your query with joins
) a
Where a.RowN = 1

Answer 2

Is the entire row duplicated exactly? If so then just add DISTINCT
SELECT DISTINCT
...
FROM
...
If you are getting duplicate rows where most columns the same but some columns are different then GROUP BY the columns that are the same and select MIN(column_name) for the ones that are causing the extra rows to appear.