This first query have one and clausle more and slow down from 200ms to 9-13seconds
i dont get why its so.
If i remove all where clause i get ~200ms just if i add one more and it will be slow.
SELECT DISTINCT a.* , p.*, p2.*, p3.*
FROM article a
LEFT JOIN pro p ON a.id = p.article_id
LEFT JOIN pro p2 ON a.id = p2.article_id
LEFT JOIN pro p3 ON a.id = p3.article_id
WHERE a.is_active = true
AND p.name = 'hotel_stars'
AND p2.name = 'article_journey_days'
AND p3.name = 'article_persons'
AND p3.int_value > 0 AND p3.int_value < 7
AND p.int_value > 0 AND p.int_value < 5
Result
319 Datensätze
Laufzeit gesamt: 9,602.081 ms
SELECT DISTINCT a.* , p.*, p2.*, p3.*
FROM article a
LEFT JOIN property p ON a.id = p.article_id
LEFT JOIN property p2 ON a.id = p2.article_id
LEFT JOIN property p3 ON a.id = p3.article_id
WHERE a.is_active = true
AND p.name = 'hotel_stars'
AND p2.name = 'article_property_journey_days'
AND p3.name = 'article_property_persons'
AND p3.int_value > 0 AND p3.int_value < 7
// AND p.int_value > 0 AND p.int_value < 5 (removed)
Result
469 Datensätze
Laufzeit gesamt: 278.453 ms
Where is the Problem?
Thx
EDIT EXPLAIN PLAN:
HashAggregate (cost=24113.80..24113.81 rows=1 width=3528)
-> Nested Loop (cost=0.00..24113.69 rows=1 width=3528)
Join Filter: (a.id = p2.article_id)
-> Nested Loop (cost=0.00..16889.70 rows=1 width=2488)
-> Nested Loop (cost=0.00..16856.58 rows=4 width=2080)
Join Filter: (p.article_id = p3.article_id)
-> Seq Scan on property p (cost=0.00..8335.87 rows=115 width=1040)
Filter: ((int_value > 0) AND (int_value < 5) AND ((name)::text = 'hotel_stars'::text))
-> Materialize (cost=0.00..8336.41 rows=107 width=1040)
-> Seq Scan on property p3 (cost=0.00..8335.87 rows=107 width=1040)
Filter: ((int_value > 0) AND (int_value < 7) AND ((name)::text = 'article_property_persons'::text))
-> Index Scan using article_pkey on article a (cost=0.00..8.27 rows=1 width=408)
Index Cond: (id = p.article_id)
Filter: is_active
-> Seq Scan on property p2 (cost=0.00..7185.05 rows=3115 width=1040)
Filter: ((name)::text = 'article_property_journey_days'::text)
16 Datensätze
Laufzeit gesamt: 11.153 ms
Changing To
SELECT DISTINCT a.* , p.*, p2.*, p3.*
FROM article a
INNER JOIN pro p ON a.id = p.article_id AND p.name = 'hotel_stars' AND p.int_value > 0 AND p.int_value < 5
INNER JOIN pro p2 ON a.id = p2.article_id AND p2.name = 'article_journey_days'
INNER JOIN pro p3 ON a.id = p3.article_id AND p3.name = 'article_persons' AND p3.int_value > 0 AND p3.int_value < 7
WHERE a.is_active = true
Result:
319 Datensätze
Laufzeit gesamt: 9,315.863 ms
HashAggregate (cost=24113.80..24113.81 rows=1 width=3528)
-> Nested Loop (cost=0.00..24113.69 rows=1 width=3528)
Join Filter: (a.id = p2.article_id)
-> Nested Loop (cost=0.00..16889.70 rows=1 width=2488)
-> Nested Loop (cost=0.00..16856.58 rows=4 width=2080)
Join Filter: (p.article_id = p3.article_id)
-> Seq Scan on property p (cost=0.00..8335.87 rows=115 width=1040)
Filter: ((int_value > 0) AND (int_value < 5) AND ((name)::text = 'hotel_stars'::text))
-> Materialize (cost=0.00..8336.41 rows=107 width=1040)
-> Seq Scan on property p3 (cost=0.00..8335.87 rows=107 width=1040)
Filter: ((int_value > 0) AND (int_value < 7) AND ((name)::text = 'article_property_persons'::text))
-> Index Scan using article_pkey on article a (cost=0.00..8.27 rows=1 width=408)
Index Cond: (id = p.article_id)
Filter: is_active
-> Seq Scan on property p2 (cost=0.00..7185.05 rows=3115 width=1040)
Filter: ((name)::text = 'article_property_journey_days'::text)
16 Datensätze
Laufzeit gesamt: 4.314 ms
Similar question :(
Added Index for column p.name and p.article.
Result speed improve from 13sec to 180ms.
Related
my query taking more than 5 minutes to execute can anyone tell me what's wrong in this query, below is the query I try to optimize it but no luck
SELECT DISTINCT
'Freight' AS Expr1000,
'Acknowledged Trip Details' AS detailtype,
sCompanyDesc,
VehicleRegistration.sVehicleRegistrationNumber,
Token.iTokenId,
sTokenNumber,
Token.dtTokenGenerating AS TokenDate,
LoadAssign.sCustomerInvoiceNumber,
dtCustomerInvoiceDate,
FillingPoint.sFillingPointName AS [From],
DecantingPoint.sDecantingPointName AS [To],
CloseLoad.fExpectedFreightAmount AS fExpectedFreightAmount,
ISNULL(PayAdvance.fAdvanceAmount, 0) AS fAdvanceAmount,
CloseLoad.fCommissionAmount AS fCommissionAmount,
CloseLoad.fDecantedQuantity,
0 AS [Shortage Quantity],
0 AS [Shortage Rate],
0 AS [Shortage Amount],
sProductName,
ISNULL((SELECT SUM(ISNULL(JVDetails.fCreditAmount, 0))
FROM Transactions
INNER JOIN JV ON JV.sJVCode = Transactions.sTransactionCode
INNER JOIN JVDetails ON JVDetails.iJVId = JV.iJVId
WHERE CloseLoad.iTokenId = Transactions.iTokenId
AND JV.sDescription = 'Munshiana'),
0) AS Munshiana,
ISNULL(JV.sJVCode, '') + ISNULL(PV.sPVCode, '') AS advdocnum
FROM CloseLoad
INNER JOIN Token ON CloseLoad.iTokenId = Token.iTokenId
INNER JOIN LoadAssign ON CloseLoad.iLoadId = LoadAssign.iLoadId
INNER JOIN FillingPoint ON Token.iFillingPointId = FillingPoint.iFillingPointId
INNER JOIN DecantingPoint ON LoadAssign.iDecantingPointId = DecantingPoint.iDecantingPointId
INNER JOIN Product ON LoadAssign.iProductId = Product.iProductId
INNER JOIN VehicleRegistration ON Token.iVehicleRegistrationId = VehicleRegistration.iVehicleRegistrationId
INNER JOIN Company ON Token.iCompanyId = Company.iCompanyId
LEFT JOIN PayAdvance ON PayAdvance.iTokenId = Token.iTokenId
AND PayAdvance.bIsApproved = 1
LEFT JOIN PV ON PV.iPayAdvanceId = PayAdvance.iPayAdvanceId
AND PayAdvance.bIsApproved = 1
AND PV.bIsApproved = 1
LEFT JOIN JV ON JV.iPayAdvanceId = PayAdvance.iPayAdvanceId
AND PayAdvance.bIsApproved = 1
AND JV.bIsApproved = 1
WHERE CloseLoad.iLoadId NOT IN (SELECT tfrli.iLoadId
FROM TempFreightReceiptLoadInfo tfrli
INNER JOIN FreightReceiptChqInfo ci ON ci.iFreightReceiptChqInfoId = tfrli.iFreightReceiptChqInfoId
INNER JOIN LoadAssign l1 ON l1.iLoadId = tfrli.iLoadId
WHERE ci.bIsApproved = 1
AND l1.dtCustomerInvoiceDate BETWEEN '8/1/2020' AND '8/30/2020'
AND tfrli.sAllocationType = 'Freight')
AND LoadAssign.bIsLoadClose = 1
AND Token.iVehicleRegistrationId = 2644
AND LoadAssign.dtCustomerInvoiceDate BETWEEN '8/1/2020' AND '8/30/2020'
AND Token.iCompanyId IN (1, 2, 3)
AND Token.sDecantingPointType IN ('Domestic', 'Export')
AND LoadAssign.bIsActive = 1
AND Token.bIsLoadVerify = 1;
Execute EXPLAIN to find bottleneck.
EXPLAIN SELECT DISTINCT ...
I have the following data schema:
With the following LINQ query:
var profiles = (
from p in context.BusinessProfiles
join u in context.Users on p.UserId equals u.Id
join addr in context.BusinessAddress on p.ProfileId equals addr.ProfileId into addrj
from addr in addrj.DefaultIfEmpty()
join pa in context.BusinessProfileActivities on p.ProfileId equals pa.ProfileId into paj
from paIfNull in paj.DefaultIfEmpty()
where p.ProfileId >= 137 && p.ProfileId <= 139
group new { p, u, addr, paIfNull }
by new {
p.ProfileId,
p.CompanyName,
p.Email,
UserEmail = u.Email,
addr.City, addr.Region,
addr.Country,
ActivityProfileId = paIfNull.ProfileId }
into pg
select new {
pg.Key.ProfileId,
pg.Key.CompanyName,
Email = pg.Key.Email ?? pg.Key.UserEmail,
pg.Key.City,
pg.Key.Region,
pg.Key.Country,
MatchingActivities = pg.Key.ActivityProfileId > 0 ? pg.Count() : 0
} into result
orderby result.MatchingActivities descending
select result
);
Which results with:
This result is corrent (ProfileId 137 has 0 activities, 138 has 1 and 139 has 2), but it produces the following SQL:
SELECT [b].[ProfileId], [b].[CompanyName], COALESCE([b].[Email], [a].[Email]) AS [Email], [b0].[City], [b0].[Region], [b0].[Country],
CASE WHEN [b1].[ProfileId] > CAST(0 AS bigint) THEN COUNT(*)
ELSE 0
END AS [MatchingActivities]
FROM [BusinessProfiles] AS [b]
INNER JOIN [AspNetUsers] AS [a] ON [b].[UserId] = [a].[Id]
LEFT JOIN [BusinessAddress] AS [b0] ON [b].[ProfileId] = [b0].[ProfileId]
LEFT JOIN [BusinessProfileActivities] AS [b1] ON [b].[ProfileId] = [b1].[ProfileId]
WHERE ([b].[ProfileId] >= CAST(137 AS bigint)) AND ([b].[ProfileId] <= CAST(139 AS bigint))
GROUP BY [b].[ProfileId], [b].[CompanyName], [b].[Email], [a].[Email], [b0].[City], [b0].[Region], [b0].[Country], [b1].[ProfileId]
ORDER BY CASE
WHEN [b1].[ProfileId] > CAST(0 AS bigint) THEN COUNT(*)
ELSE 0
END DESC
In SQL, I can avoid both CASE WHEN if I use COUNT([b1].[ProfileId]) like this:
SELECT [b].[ProfileId], [b].[CompanyName], COALESCE([b].[Email], [a].[Email]) AS [Email], [b0].[City], [b0].[Region], [b0].[Country],
COUNT([b1].[ProfileId]) AS [MatchingActivities]
FROM [BusinessProfiles] AS [b]
INNER JOIN [AspNetUsers] AS [a] ON [b].[UserId] = [a].[Id]
LEFT JOIN [BusinessAddress] AS [b0] ON [b].[ProfileId] = [b0].[ProfileId]
LEFT JOIN [BusinessProfileActivities] AS [b1] ON [b].[ProfileId] = [b1].[ProfileId]
WHERE ([b].[ProfileId] >= CAST(137 AS bigint)) AND ([b].[ProfileId] <= CAST(139 AS bigint))
GROUP BY [b].[ProfileId], [b].[CompanyName], [b].[Email], [a].[Email], [b0].[City], [b0].[Region], [b0].[Country], [b1].[ProfileId]
ORDER BY [MatchingActivities] DESC
My question is, how can I count by grouped ActivityProfileId = paIfNull.ProfileId using LINQ and get EF to generate the above SQL?
I have tried so many variations resulting mostly in EF to SQL errors.
MatchingActivities = pg.Count(t => t.ActivityProfileId!= 0)
MatchingActivities = pg.Select(t => t.paIfNull.ProfileId).Distinct().Count(),
MatchingActivities = pg.Count(t => t.paIfNull != null),
All result in errors like System.InvalidOperationException: The LINQ expression ... could not be translated. or getting MatchingActivities as 1 instead of 0.
Related Q/A:
LINQ Count returning 1 instead of zero for an empty group
Group by in LINQ
How to write left join, group by and average in c# entity framework Linq
In short you can't! EF Core still doesn't support that.
See this:
https://github.com/dotnet/efcore/issues/17376
And also See:
https://stackoverflow.com/a/61878332/9212040
My select statement returns the ids from the tables I have joined correctly(393, starting at 92 and going up 484, its not a PK field, so some ids can be used more then once) there is a total of 550 rows I need to update, when I run the update it only adds the number 92 to the table for all 550 rows.
update movements set [location] = locaID.id FROM (
SELECT
lo.id
FROM [Harvest_Transactions] ht
left join [Harvest_Master_Shift] hms on ht.Harvest_Master_id = hms.Harvest_Master_id
left join [Greenhouse_Troughs] gt on ht.Trough = gt.Greenhouse_Trough_id
left join [batches] b on b.name = hms.Batch_id
left join [phases] p on p.batch = b.id and p.[type] = 3
left join #loc lo on lo.name = gt.Trough_No and lo.area = gt.SQM_per_Trough and lo.Bay_id = gt.Bay_id
where ht.Number_of_Plants_Harvested != 0 and (hms.CreatedOn <= '2020-02-05 09:33:00.000' OR hms.CreatedOn is null )
)locaID where product = 14
what am I missing so it updates with the correct values?
My first impression of your query is that only the rows with product = 14 in movements table will be updated with the single value coming from the subquery.
If you want to update the values based on another derived table, you need some way to link the rows together - i.e a JOIN between your table being updated and the table that holds the other data (see here )
What is the common column between movements and
SELECT
lo.id
FROM [Harvest_Transactions] ht
left join [Harvest_Master_Shift] hms on ht.Harvest_Master_id = hms.Harvest_Master_id
left join [Greenhouse_Troughs] gt on ht.Trough = gt.Greenhouse_Trough_id
left join [batches] b on b.name = hms.Batch_id
left join [phases] p on p.batch = b.id and p.[type] = 3
left join #loc lo on lo.name = gt.Trough_No and lo.area = gt.SQM_per_Trough and
lo.Bay_id = gt.Bay_id
where ht.Number_of_Plants_Harvested != 0 and (hms.CreatedOn <= '2020-02-05
09:33:00.000' OR hms.CreatedOn is null )
The rough syntax shouldI think be
UPDATE movements
set [location] = locaID.id
FROM movements
JOIN (
SELECT
lo.id
FROM [Harvest_Transactions] ht
left join [Harvest_Master_Shift] hms on ht.Harvest_Master_id = hms.Harvest_Master_id
left join [Greenhouse_Troughs] gt on ht.Trough = gt.Greenhouse_Trough_id
left join [batches] b on b.name = hms.Batch_id
left join [phases] p on p.batch = b.id and p.[type] = 3
left join #loc lo on lo.name = gt.Trough_No and lo.area = gt.SQM_per_Trough and
lo.Bay_id = gt.Bay_id
where ht.Number_of_Plants_Harvested != 0 and (hms.CreatedOn <= '2020-02-05
09:33:00.000' OR hms.CreatedOn is null )
) locaID
ON movements.<some column> = locaID.<some column>
I have this table:
| id_product | tw_kod | ean |nazwa |stan |cena_detaliczna |
| 1 | 9007 | 9008 | ARR 2016 | 2 | 2490|
| 2 | 9008 | 90085| ION 2018 | NULL | 1450|
and this query:
select
tw.id as id_product,
tw.kod as tw_kod,
tw.kod as ean,
tw.nazwa as nazwa,
sm.stanHandl as stan,
cn.cena as cena_detaliczna
from
[HM].[TW] as tw
join
[SSCommon].[STElements] as elem on tw.znacznik = elem.Shortcut
left join
[HM].[SM] as sm on tw.id = sm.idtw
left join
[HM].[CN] as cn on tw.id = cn.idpm
where
elem.Shortcut = '66'
and elem.ElementKindId = '54'
--and sm.magazyn = 2
and cn.typceny = 1
order by
nazwa
Some products were in [HM].[SM] table, some - not. and I use LEFT JOIN for [HM].[SM] to show all products in [HM].[TW], but when I use WHERE clause (sm.magazyn= 2) it doesn't show me products, that's why I commented it. I tried to use IF ELSE in WHERE, but I had not correct result. Thanks
Just add predicate in FROM section like this:
select
tw.id as id_product,
tw.kod as tw_kod,
tw.kod as ean,
tw.nazwa as nazwa,
sm.stanHandl as stan,
cn.cena as cena_detaliczna
from [HM].[TW] as tw
join [SSCommon].[STElements] as elem on tw.znacznik = elem.Shortcut
left join [HM].[SM] as sm on tw.id = sm.idtw and sm.magazyn = 2
left join [HM].[CN] as cn on tw.id = cn.idpm and cn.typceny = 1
where
elem.Shortcut = '66'
and elem.ElementKindId = '54'
order by nazwa
Compassion with NULL causes empty resultset
So I have two subqueries that return the same columns from the same table
Query #1:
SELECT E.Id,E.Title,E.LocationId,P.LocationId,E.DepartmentId,P.DepartmentId,E.DateCreated,E.IsActive,E.IsHotJob,E.RequisitionId,E.RequisitionIdString,E.RewardSettingId,E.EmploymentOpportunityStatusId
FROM EmploymentOpportunities E, Profiles P
WHERE E.EmploymentOpportunityStatusId = 9 AND E.IsActive = 1 AND E.IsHotjob = 1
AND P.Id = 'C5F07EBB-CE81-4133-A462-241A5F84D418' AND (P.DepartmentId != E.DepartmentId AND P.LocationId != E.LocationId)
ORDER BY E.DateCreated DESC
Query #2:
SELECT E.Id,E.Title,E.LocationId,P.LocationId,E.DepartmentId,P.DepartmentId,E.DateCreated,E.IsActive,E.IsHotJob,E.RequisitionId,E.RequisitionIdString,E.RewardSettingId,E.EmploymentOpportunityStatusId
FROM EmploymentOpportunities E, Profiles P
WHERE E.EmploymentOpportunityStatusId = 9 AND E.IsActive = 1 AND E.IsHotjob = 0 AND
P.Id = 'C5F07EBB-CE81-4133-A462-241A5F84D418' AND (P.DepartmentId = E.DepartmentId OR P.LocationId = E.LocationId)
ORDER BY E.DateCreated DESC
I want this two queries combines into one but preserve the order they have, so somehow stack Query #1 onto Query #2.
Is this possible?
SELECT 1 SetNumber, E.Id,E.Title,E.LocationId,P.LocationId,E.DepartmentId,
P.DepartmentId,E.DateCreated,E.IsActive,E.IsHotJob,E.RequisitionId,
E.RequisitionIdString,E.RewardSettingId,E.EmploymentOpportunityStatusId
FROM EmploymentOpportunities E, Profiles P
WHERE E.EmploymentOpportunityStatusId = 9 AND E.IsActive = 1
AND E.IsHotjob = 1 P.Id = 'C5F07EBB-CE81-4133-A462-241A5F84D418'
AND (P.DepartmentId != E.DepartmentId AND P.LocationId != E.LocationId)
union all
SELECT 2, E.Id,E.Title,E.LocationId,P.LocationId,E.DepartmentId,P.DepartmentId,
E.DateCreated,E.IsActive,E.IsHotJob,E.RequisitionId,E.RequisitionIdString,
E.RewardSettingId,E.EmploymentOpportunityStatusId
FROM EmploymentOpportunities E, Profiles P
WHERE E.EmploymentOpportunityStatusId = 9 AND E.IsActive = 1
AND E.IsHotjob = 0 AND P.Id = 'C5F07EBB-CE81-4133-A462-241A5F84D418'
AND (P.DepartmentId = E.DepartmentId OR P.LocationId = E.LocationId)
ORDER BY SetNumber, DateCreated desc
Just use UNION ALL
SELECT * FROM
(
Your Query1
UNION ALL // do not use order by here
Your Query2
) AS someName
ORDER BY yourColumn DESC
You can also use UNION - but it will filter out duplicates if any.