How to identify why rows are duplicating - sql-server

The below query selects all the rows from a master table (there are no duplicates in the table i have checked) and this query counts the number of business days and checks it against a certain number of days, if the count is greater than that certain number of days, it is late, else it is not late. For some reason when I run this query, even with the select DISTINCT, my rows are duplicating. I see a row for each tracking number showing one as Late and the other as Not Late but I don't see how they could possibly be showing both answers? Could anyone help me understand why my rows are being duplicated?
SELECT DISTINCT case UT.[Service] when '0PW' then 'UPS SurePost 1 lb or Greater' else UT.[Service] end as 'Service'
,US.[Region]
,UT.[Tracking_Number] as 'TrackingCounts'
,UT.Manifest_Date
,UT.Date_Delivered
,Ship_To_Postal_Code
,WarehouseLocation
,CASE
WHEN UT.[Service] = 'UPS Ground' AND WarehouseLocation = 'Wausau' then iif(((
DATEDIFF(dd, UT.[Manifest_Date], CASE WHEN (UT.[Date_Delivered] = '1/1/2099') THEN GETDATE() WHEN (UT.[Date_Delivered] IS NULL) THEN GETDATE() ELSE UT.[DATE_DELIVERED] END) )
-(DATEDIFF(wk, UT.[Manifest_Date], CASE WHEN (UT.[Date_Delivered] = '1/1/2099') THEN GETDATE() WHEN (UT.[Date_Delivered] IS NULL) THEN GETDATE() ELSE UT.[DATE_DELIVERED] END) * 2)
-(CASE WHEN DATENAME(dw, UT.[Manifest_Date]) = 'Sunday' THEN 1 ELSE 0 END)
-(CASE WHEN DATENAME(dw, UT.[Date_Delivered]) = 'Saturday' THEN 1 ELSE 0 END)
-(Select Count(*) from [Reporting_Operations].[CORP\u694172].[tbl_HolidayDates] where [HolidayDate] between UT.[Manifest_Date] and UT.[Date_Delivered])) > UZ.TNTDAYS, 1, 0)
WHEN UT.[Service] in ('UPS SurePost 1 lb or Greater','UPS SurePost Less than 1 lb','0PW') AND WarehouseLocation = 'Wausau' then iif(((
DATEDIFF(dd, UT.[Manifest_Date], CASE WHEN (UT.[Date_Delivered] = '1/1/2099') THEN GETDATE() WHEN (UT.[Date_Delivered] IS NULL) THEN GETDATE() ELSE UT.[DATE_DELIVERED] END) )
-(DATEDIFF(wk, UT.[Manifest_Date], CASE WHEN (UT.[Date_Delivered] = '1/1/2099') THEN GETDATE() WHEN (UT.[Date_Delivered] IS NULL) THEN GETDATE() ELSE UT.[DATE_DELIVERED] END) * 2)
-(CASE WHEN DATENAME(dw, UT.[Manifest_Date]) = 'Sunday' THEN 1 ELSE 0 END)
-(CASE WHEN DATENAME(dw, UT.[Date_Delivered]) = 'Saturday' THEN 1 ELSE 0 END)
-(Select Count(*) from [Reporting_Operations].[CORP\u694172].[tbl_HolidayDates] where [HolidayDate] between UT.[Manifest_Date] and UT.[Date_Delivered])) > (UZ.TNTDAYS + 1), 1, 0)
ELSE iif(((
DATEDIFF(dd, UT.[Manifest_Date], CASE WHEN (UT.[Date_Delivered] = '1/1/2099') THEN GETDATE() WHEN (UT.[Date_Delivered] IS NULL) THEN GETDATE() ELSE UT.[DATE_DELIVERED] END))
-(DATEDIFF(wk, UT.[Manifest_Date], CASE WHEN (UT.[Date_Delivered] = '1/1/2099') THEN GETDATE() WHEN (UT.[Date_Delivered] IS NULL) THEN GETDATE() ELSE UT.[DATE_DELIVERED] END) * 2)
-(CASE WHEN DATENAME(dw, UT.[Manifest_Date]) = 'Sunday' THEN 1 ELSE 0 END)
-(CASE WHEN DATENAME(dw, UT.[Date_Delivered]) = 'Saturday' THEN 1 ELSE 0 END)
-(Select Count(*) from [Reporting_Operations].[CORP\u694172].[tbl_HolidayDates] where [HolidayDate] between UT.[Manifest_Date] and UT.[Date_Delivered])) > USS.[Days], 1, 0)
End as LATE
FROM [Reporting_Operations].[CORP\u694172].[tbl_UPS_Automation_QVM_Tracking] UT
INNER JOIN [Reporting_Operations].[CORP\u694172].[tbl_UPS_Automation_States] US on US.[State] = UT.[Ship_To_State_Province]
INNER JOIN [Reporting_Operations].[CORP\u694172].[tbl_UPS_Automation_Ground_Zips] UZ on UZ.[DESTZIPCODE] = UT.[Ship_To_Postal_Code]
INNER JOIN [Reporting_Operations].[CORP\u694172].[tbl_UPS_Automation_Services] USS on UT.[Service] = UT.[Service]
where cast(UT.Manifest_Date as date) between '1/20/2019' and '1/26/2019' AND WarehouseLocation = 'Wausau' and UT.Status = 'Delivered'
group by UT.[Service],US.[Region], USS.Days, UT.[Manifest_Date], UT.[Date_Delivered], TNTDAYS, Tracking_Number, Ship_To_Postal_Code, WarehouseLocation
So I found the issue is on INNER JOIN [Reporting_Operations].[CORP\u694172].[tbl_UPS_Automation_Services] USS on UT.[Service] = UT.[Service]
however I don't understand why when i join UT.SERVICE on USS.Service the uss.service field shows every possible service, and not the value that Ut.SERVICE has.
if UT.Service = UPS 2nd Day Air wouldn't the join on USS only give me UPS 2nd Day Air values?

The root cause is one of the joins returns more than 1 records
The way you need to find the the root cause of issue is:
1- Remove all columns from your select to become Select * from yourtable
2- comment all joins
3- uncomment joins and watch your data to see if the become duplicate
3.1- if the records are not duplicated that join is OK, do the same on next join
3.2- if records get duplicated, means join might not be right, usually some join conditions are missing
4- After fixing the join issue, remove * and bring back the fields
5- do final check

Related

previous day's count except on Monday, use Friday's counts

I have the below code. I am comparing counts from today's date to yesterday's date unless date falls on Monday then I want to use Friday's count.
select
cast(getdate() as date)run_dt,
'dx' as db,
'dbo' as [schema],
'tb_cust' as [table],
'completed status' as check_type,
a.curr_cnt,
a.prev_cnt,
abs(cast((a.prev_cnt - a.curr_cnt) / cast(a.curr_cnt as decimal(10,2)) as decimal(5,2))*100) as cal_prcnt_dff,
case
when abs(cast((a.prev_cnt - a.curr_cnt) / cast(a.curr_cnt as decimal(10,2)) * 100 as int) ) >= 100
then 'Y'
else 'N'
end email_trigger,
concat('count on',' ', #curr_dt,' ','[',datename(dw,#curr_dt),']',' ' ,'increased/decreased by 100% from',' ',#prev_dt,' ','[',datename(dw,#prev_dt),']') as 'comment'
from (select
sum(
case
when a.custstatus = 'completed'
and a.custdate = cast(getdate() -1 as date)
then 1
else 0
end) curr_cnt,
sum(
case
when a.custstatus = 'completed'
and a.custtdate = cast(getdate() -2 as date)
--and cast(getdate() as date) in ()
--and datepart(dw,getdate()) in (3,4,5,6)
--or
then 1
else 0
end) prev_cnt,
a.custstatus
from cte_apt a
where 1=1
and a.custstatus = 'completed'
group by
a.custstatus)a
How can I get prev_cnt to count Friday when the day lands on a Monday?
sum(
case
when a.custstatus = 'completed'
and a.custtdate = cast(getdate() -2 as date)
--and cast(getdate() as date) in ()
--and datepart(dw,getdate()) in (3,4,5,6)
--or
then 1
else 0
end) prev_cnt,

Count Number of Saturdays in a date range - SQL

I have a report which looks for orders during a given date range... It returns the DateName from the report to give me mondays, it then gives me times from the report, to give me 859 for 08:59 for example.. I then use a case on this report to do the following...
WHEN (DATENAME(DW,T1.DocDate)) = 'Monday' AND T1.DocTime >= '700' AND T1.DocTime <= '859' THEN '1 Monday 07:00-08:59'
What I want to achieve, is the count for "1 Monday 07:00-08:59" to be divided by 2 if there has been 2 mondays, divided by 3 if there has been 3 mondays etc... but I have to be able to have it divide by 3 mondays for example, but 2 wednesdays if the date is a tuesday...
The report currently gives a Total of all orders placed in a datetime grouped together but has no division to average for each day.
SELECT (DATENAME(DW, T1.DocDate)) AS Weekday,
T1.DocTime AS Time,
SUM(T1.DocTotal) AS Value,
CASE
WHEN (DATENAME(DW, T1.DocDate)) = 'Monday'
AND T1.DocTime >= '700'
AND T1.DocTime <= '859' THEN '1 Monday 07:00-08:59'
FROM ORDR T1
INNER JOIN OCRD T0 ON T0.CardCode = T1.CardCode
WHERE (T1.DocDate >= #Start
AND T1.DocDate <= #End)
AND T0.QryGroup20 = 'Y'
AND T1.Canceled = 'N'
GROUP BY T1.DocTime,
T1.DocDate;
I expect the report to count the mondays / tuesdays etc that occur between #START and #END to give me an average sale per day and time.Then be able to then somehow divide by the count of the defined days in the case statement. (I'll hopefully be doing this part in Crystal Reports) but if i can get the first part, I'll work on the second half.
Just cross join the DW_Count subquery, which returns the count of every weekday in your orders range. Then you can count your avg for each weekday
SELECT (DATENAME(DW, T1.DocDate)) AS Weekday,
T1.DocTime AS Time,
SUM(T1.DocTotal) AS Value,
SUM(T1.DocTotal)/(CASE WHEN DATENAME(DW, T1.DocDate) = 'Monday' then DW_Count.MonCount
WHEN DATENAME(DW, T1.DocDate) = 'Tuesday' then DW_Count.TueCount
WHEN DATENAME(DW, T1.DocDate) = 'Wednesday' then DW_Count.WedCount
WHEN DATENAME(DW, T1.DocDate) = 'Thursday' then DW_Count.ThuCount
WHEN DATENAME(DW, T1.DocDate) = 'Friday' then DW_Count.FriCount
WHEN DATENAME(DW, T1.DocDate) = 'Saturday' then DW_Count.SatCount
WHEN DATENAME(DW, T1.DocDate) = 'Sunday' then DW_Count.SunCount) AS AvgValue
CASE
WHEN (DATENAME(DW, T1.DocDate)) = 'Monday'
AND T1.DocTime >= '700'
AND T1.DocTime <= '859' THEN '1 Monday 07:00-08:59'
FROM ORDR T1
INNER JOIN OCRD T0 ON T0.CardCode = T1.CardCode
CROSS JOIN (select sum(case when DATENAME(DW, DW_Count.DateValue) = 'Monday' then 1 else 0 end) as MonCount,
sum(case when DATENAME(DW, DW_Count.DateValue) = 'Tuesday' then 1 else 0 end) as TueCount,
sum(case when DATENAME(DW, DW_Count.DateValue) = 'Wednesday' then 1 else 0 end) as WedCount,
sum(case when DATENAME(DW, DW_Count.DateValue) = 'Thursday' then 1 else 0 end) as ThuCount,
sum(case when DATENAME(DW, DW_Count.DateValue) = 'Friday' then 1 else 0 end) as FriCount,
sum(case when DATENAME(DW, DW_Count.DateValue) = 'Saturday' then 1 else 0 end) as SatCount,
sum(case when DATENAME(DW, DW_Count.DateValue) = 'Sunday' then 1 else 0 end) as SunCount
from (select distinct DocDate from ORDR where (T1.DocDate >= #Start AND T1.DocDate <= #End))) AS DW_Count
WHERE (T1.DocDate >= #Start
AND T1.DocDate <= #End)
AND T0.QryGroup20 = 'Y'
AND T1.Canceled = 'N'
GROUP BY T1.DocTime,
T1.DocDate;
You can use the below logic-
For MSSQL
DECLARE #DT1 DATE = '20190802'
DECLARE #DT2 DATE = '20190820'
DECLARE #SatCount INT = 0
WHILE #DT1<= #DT2
BEGIN
SET #SatCount = #SatCount + CASE WHEN DATEPART(WeekDay,#DT1) = 7 THEN 1 ELSE 0 END
SET #DT1 = DATEADD(DD,1,#DT1)
END
SELECT #SatCount
Just one more option-
DECLARE #DT1 DATE = '20190802',
#DT2 DATE = '20190920';
SELECT SUM(CASE WHEN DATEPART(WeekDay,Date) = 7 THEN 1 ELSE 0 END)
FROM
(
SELECT TOP (DATEDIFF(DAY, #DT1, #DT2) + 1)
Date = DATEADD(DAY, ROW_NUMBER() OVER(ORDER BY a.object_id) - 1, #DT1)
FROM sys.all_objects a
-- A System table just used for creating multiple rows.
CROSS JOIN sys.all_objects b
-- CROSS JOIN will create number of row = original number of row X original number of row.
)A

Simplifying Query to make it run faster

I have this extremely long winded query below. I am having issues with running it as it takes forever and keeps timing out on me:
with t as
(
select a.ID,
a.Date_Reported AS [Date Sent],
b.Date_Received AS [Date Returned],
(datediff(dd, a.date_reported, b.date_received)
+ CASE WHEN Datepart(dw, b.date_received) = 7 THEN 1 ELSE 0 END
- (Datediff(wk, a.date_reported, b.date_received) * 2 )
- CASE WHEN Datepart(dw, b.date_received) = 1 THEN 1 ELSE 0 END +
- CASE WHEN Datepart(dw, b.date_received) = 1 THEN 1 ELSE 0
END) AS [Overall_Time_Spent]
from [Transactions_External] a
join [Transactions] b on b.id like '%'+a.id+'%'
where a.customer = 'AA'
AND a.Date_Reported >= DATEADD(MONTH,-1,DATEADD(MONTH,DATEDIFF(MONTH,0,GETDATE()),0))
AND a.Date_Reported < DATEADD(d,1,EOMONTH(GETDATE(),-1))
AND a.ID IS NOT NULL
AND a.ID <> ''
AND b.ID not like '%_H'
)
select V.*
from
(
select
sum(case when Overall_Time_Spent < 0 then 1 else 0 end) as Errors,
sum(case when Overall_Time_Spent between 0 and 3 then 1 else 0 end) as _0_3_days,
sum(case when Overall_Time_Spent = 4 then 1 else 0 end) as _4_days,
sum(case when Overall_Time_Spent = 5 then 1 else 0 end) as _5_days,
sum(case when Overall_Time_Spent between 6 and 8 then 1 else 0 end) as _6_8_days,
sum(case when Overall_Time_Spent >= 9 then 1 else 0 end) as more_than_9_days,
count(Overall_Time_Spent) as Total
from t
) T1
cross apply
( values
('Count', convert(int, [Errors]), convert(int, [_0_3_days]), convert(int, [_4_days]), convert(int, [_5_days]), convert(int, [_6_8_days]), convert(int, [more_than_9_days]), convert(int, [Total]))
)
v([Time Taken (days)], [Errors], [0-3],[4],[5],[6-8],[9+], [Total])
The query is essentially looking at two tables, joining on id (which is slightly different on either table hence the like on the join) and then finding the difference in two dates to find the overall time spent. Then later on the times are split up into ranges. The query is restricted to last month only.
Any ideas what I can do to make this run faster or change the query about to help it run faster. I think the issue may be in the original select:
datediff(dd, a.date_reported, b.date_received)
+ CASE WHEN Datepart(dw, b.date_received) = 7 THEN 1 ELSE 0 END
- (Datediff(wk, a.date_reported, b.date_received) * 2 )
- CASE WHEN Datepart(dw, b.date_received) = 1 THEN 1 ELSE 0 END +
- CASE WHEN Datepart(dw, b.date_received) = 1 THEN 1 ELSE 0
END) AS [Overall_Time_Spent]
I may be selecting on all the database rather than last month?
one important thing to note is i am unable to create any tables or split the query up- so i really need to run selects and do it in one query. I am not sure this is possible.
join with like and "%" first is not recommended
join [Transactions] b on b.id like '%'+a.id+'%'
Index will not be used on a.id (if any) and it would require full scan. Maybe try to do an EXPLAIN of your query to see number of row scanned

Last 2 month record in SQL

I try this SQL query to get current total and also last 2 months records and current month record means total 3 months..
Select distinct
tblRv.Owner,
(Select Count(*) as total from tblvv WHERE MasterID =tblRv.ID and Name <> '')
as currentdata
from tblRe
inner join tblRv
On tblRe.RID = tblRv.RID
WHERE
tblRe.StartDate between dateadd(m, -2, getdate()) and getdate() and
//tblRe.StartDate >= '2016-07-01 00:00:00' AND
//tblRe.EndDate <= '2016-07-08 23:59:59'
and tblRe.Region = 'uk' and
tblRv.Owner='Roh'
order by tblRv.Owner
when i exe this show me like this
OwnerName currentdata
Roh 1
Roh 2
Roh 3
Roh 5
and when i check individually write query and check from date 2016-07-01 and todate 2016-07-30 dates then this show me data 3 and 2016-06-01 00:00:00 and 2016-06-31 23:59:59 show me data 1 and 2016-05-01 00:00:00 ,2016-05-31 23:59:59 show me data 0
so i want data like this
owner july june may
roh 3 1 0
also when there will be current month i.e. aug then data must display last 2 months i.e.june july
Select
tblRv.Owner
,DATENAME(MONTH,tblRe.StartDate) as [Month]
,ISNULL(SUM(total),0) as currentdata
from tblRe
INNER JOIN tblRv ON tblRe.RID = tblRv.RID
LEFT JOIN (
Select Count(*) as total ,MasterID
from tblvv
WHERE Name <> ''
GROUP BY MasterID
) tblvv
ON tblvv.MasterID =tblRv.ID
WHERE tblRe.StartDate >= DATEADD(MONTH, -2, GETDATE())
AND tblRe.EndDate <= GETDATE()
AND tblRe.Region = 'uk'
AND tblRv.[Owner] ='Roh'
GROUP BY tblRv.Owner
,DATENAME(MONTH,tblRe.StartDate)
order by tblRv.[Owner]
You can try doing a pivot query where the three columns to be pivoted are the current, previous, and previous previous month in your data set. Note that I had to rewrite your query to remove the subqueries in the SELECT clause, because this makes it impossible to do aggregation of those columns.
SELECT tblRv.Owner,
SUM(CASE WHEN tblRe.StartDate = GETDATE() THEN t.total ELSE 0 END) AS currMonth,
SUM(CASE WHEN tblRe.StartDate = DATEADD(m, -1, GETDATE())
THEN t.total ELSE 0 END) AS prevMonth,
SUM(CASE WHEN tblRe.StartDate = DATEADD(m, -2, GETDATE())
THEN t.total ELSE 0 END) AS lastPrevMonth
FROM tblRe
INNER JOIN tblRv
ON tblRe.RID = tblRv.RID
INNER JOIN
(
SELECT MasterID, COUNT(*) AS total
FROM tblvv
GROUP BY MasterID
WHERE Name <> ''
) AS t
ON tblRv.ID = t.MasterID
WHERE tblRe.StartDate BETWEEN DATEADD(m, -2, GETDATE()) AND GETDATE() AND
tblRe.Region = 'uk' AND
tblRv.Owner = 'Roh'
GROUP BY tblRv.Owner
ORDER BY tblRv.Owner

How to add Totals in SQL

I am trying to get the totals of each month as of YTD (Years to date) Can someone please help me figure out how to integrate this in my query? thank you This is what I have so far.
DECLARE #Year int
set #Year = 2013
select a.first_name, a.last_name
, COUNT(CASE WHEN MONTH(b.Funded_date) = 1 THEN 1 ELSE NULL END) January
, COUNT(CASE WHEN MONTH(b.Funded_date) = 2 THEN 1 ELSE NULL END) February
, COUNT(CASE WHEN MONTH(b.Funded_date) = 3 THEN 1 ELSE NULL END) March
, COUNT(CASE WHEN MONTH(b.Funded_date) = 4 THEN 1 ELSE NULL END) April
From tContact a Join tContract b ON a.contact_id = b.contract_id
Group by a.first_name, a.last_name
This is just an example of how you could count up rows that fall under a certain month.
SELECT MONTH(b.Funded_date) AS 'MonthNum',
COUNT(*) AS 'Total'
FROM Table AS b
WHERE YEAR(b.Funded_date) = 2014
GROUP BY MONTH(b.Funded_date)
Hopefully this will help you with your query.
Thanks
What I tried to do here is create an upper bound record for each month in tContract then join that back into the query you already had. It is joined on dates that are between the beginning of the year and the current month.
DECLARE #Year int
set #Year = 2013
select Ms.thismonth, count(B.thing_You_are_Totalling) from (
select thisMonth = dateadd(month,datediff(month,0,Funded_date),0)
from tContract
where moid = 2005405
and year(Funded_date) = #Year
group by dateadd(month,datediff(month,0,Funded_date),0)
) Ms
inner join (select * from tContact a inner join tContract ON a.contact_id = tContract.contract_id) B
on B.Funded_date >=dateadd(year,datediff(year,0,B.Funded_date),0) -- beginning of year
and B.Funded_date <= Ms.thisMonth -- this month
where year(B.Funded_date) = #Year -- restrict to only this year
group by thisMonth, first_name, last_name
I don't have your full table definition so there might be some issues (maybe a SqlFiddle is in order)
Not sure if this is what you are asking for.
select a.first_name, a.last_name
, COUNT(CASE WHEN MONTH(b.Funded_date) = 1 THEN 1 ELSE NULL END) January
, COUNT(CASE WHEN MONTH(b.Funded_date) = 2 THEN 1 ELSE NULL END) February
, COUNT(CASE WHEN MONTH(b.Funded_date) = 3 THEN 1 ELSE NULL END) March
, COUNT(CASE WHEN MONTH(b.Funded_date) = 4 THEN 1 ELSE NULL END) April
, COUNT(*) TotalCount
, SUM(CASE WHEN MONTH(b.Funded_date) = 1 THEN Amount ELSE NULL END) JanuaryAmount
, SUM(CASE WHEN MONTH(b.Funded_date) = 2 THEN Amount ELSE NULL END) FebruaryAmount
, SUM(CASE WHEN MONTH(b.Funded_date) = 3 THEN Amount ELSE NULL END) MarchAmount
, SUM(CASE WHEN MONTH(b.Funded_date) = 4 THEN Amount ELSE NULL END) AprilAmount
From tContact a Join tContract b ON a.contact_id = b.contact_id
WHERE YEAR(b.Funded_date) = #Year
Group by a.first_name, a.last_name
How about this:
declare #year int = 2013
select a.first_name
, a.last_name
, month(b.Funded_date) [Month]
, datename(month, dateadd(month, month(date_of_birth_dt), - 1)) [MonthName]
, count(month(b.Funded_date)) [Total]
from tContact a
where a.[Year] = #year
group by a.first_name, a.last_name, month(b.Funded_date)
It returns the total of each Month for the year 2013. a.[Year] might not the the name of the field that you have so adjust accordingly. Also, [Month] returns numeric value for month.
Use the Count(*) As Total function. I'm sure this will help you
SELECT MONTH(b.Funded_date) AS 'MonthNum',
COUNT(*) AS 'Total'
FROM Table AS b
WHERE YEAR(b.Funded_date) = 2014
GROUP BY MONTH(b.Funded_date)

Resources