I am using EF Core for my .NET Core 2.1 project to interact with Azure SQL Server.
I have query which fetches around 1.5L rows by joining multiple tables. It's taking a long time to return the result and it is getting timed out (more than 30 seconds). So I am going with pagination using skip and take.
So my final call looks like
query.Skip((pageNumber - 1) * pageSize).Take(pageSize).ToList();
This call works absolutely fine and give the expected results.
But when I add order by clause to this query:
query.OrderBy(sortField)
.Skip((pageNumber - 1) * pageSize).Take(pageSize).ToList()
Below is the generated SQL
SELECT [p].[Id], [p].[CreatedById], [p].[CreatedOn], [p].[CurrencyId], [p].[CustomerId], [p].[DataMigrationLogId], [p].[FollowUp], [p].[IsActive], [p].[ProjectName], [p].[PlantCode], [p].[ShipToDistanceFromPlant], [p].[StatusId],
[p].[UpdatedById], [p].[UpdatedOn], [p.DataMigrationLog].[Id], [p.DataMigrationLog].[CreatedOn], [p.DataMigrationLog].[GeneratedOn], [p.DataMigrationLog].[HasEdsBid], [p.DataMigrationLog].[HasMbiBid], [p.DataMigrationLog].[Log],
[p.DataMigrationLog].[RequestXml], [p.DataMigrationLog].[Status], [p.DataMigrationLog].[Xq1ProjectId], [p.UpdatedBy].[Id], [p.UpdatedBy].[CellPhone], [p.UpdatedBy].[CreatedOn], [p.UpdatedBy].[Discriminator], [p.UpdatedBy].[Email],
[p.UpdatedBy].[FirstName], [p.UpdatedBy].[HasAccessToDoors], [p.UpdatedBy].[HasAccessToWindows], [p.UpdatedBy].[IsActive], [p.UpdatedBy].[LastName], [p.UpdatedBy].[Prefix], [p.UpdatedBy].[RecentProjectId], [p.UpdatedBy].[UpdatedOn],
[p.UpdatedBy].[WorkPhone], [p.UpdatedBy].[XQ1LoginName], [p.UpdatedBy].[IsSuperAdmin], [p.UpdatedBy].[HasAccessToRestrictedReports], [p.UpdatedBy].[HasEdsProgramAndDealerPriceAccess], [p.UpdatedBy].[IsIss],
[p.UpdatedBy].[ProjectVisibilityId], [p.UpdatedBy].[CustomerId], [p.UpdatedBy].[IsApiUser], [p.CreatedBy].[Id], [p.CreatedBy].[CellPhone], [p.CreatedBy].[CreatedOn], [p.CreatedBy].[Discriminator], [p.CreatedBy].[Email],
[p.CreatedBy].[FirstName], [p.CreatedBy].[HasAccessToDoors], [p.CreatedBy].[HasAccessToWindows], [p.CreatedBy].[IsActive], [p.CreatedBy].[LastName], [p.CreatedBy].[Prefix], [p.CreatedBy].[RecentProjectId], [p.CreatedBy].[UpdatedOn],
[p.CreatedBy].[WorkPhone], [p.CreatedBy].[XQ1LoginName], [p.CreatedBy].[IsSuperAdmin], [p.CreatedBy].[HasAccessToRestrictedReports], [p.CreatedBy].[HasEdsProgramAndDealerPriceAccess], [p.CreatedBy].[IsIss],
[p.CreatedBy].[ProjectVisibilityId], [p.CreatedBy].[CustomerId], [p.CreatedBy].[IsApiUser], [p.Status].[Id], [p.Status].[Code], [p.Status].[Name], [p.Customer].[Id], [p.Customer].[AxCustomerNumber], [p.Customer].[CreatedById],
[p.Customer].[CreatedOn], [p.Customer].[CreditTermId], [p.Customer].[CurrencyId], [p.Customer].[CustomerTypeId], [p.Customer].[DefaultPricing], [p.Customer].[FabricSystemFreight], [p.Customer].[IsActive], [p.Customer].[IsSpecialFreight],
[p.Customer].[LocalityRepId], [p.Customer].[MinFreightCharge], [p.Customer].[CompanyName], [p.Customer].[Notes], [p.Customer].[PrimaryBusinessId], [p.Customer].[ProgramAccountId], [p.Customer].[Prospect],
[p.Customer].[ShippingDollarsThreshold], [p.Customer].[ShippingMilesThreshold], [p.Customer].[SpecialNote], [p.Customer].[SpecialPlantInstruction], [p.Customer].[UpdatedById], [p.Customer].[UpdatedOn],
[p.Customer].[Xq1ProgGroupName], [p.Customer].[Xq1SrNo], [p.Customer].[XqCustomerNumber], [p.Customer.ProgramAccount].[Id], [p.Customer.ProgramAccount].[AxCustomerNumber], [p.Customer.ProgramAccount].[CreatedById],
[p.Customer.ProgramAccount].[CreatedOn], [p.Customer.ProgramAccount].[CreditTermId], [p.Customer.ProgramAccount].[CurrencyId], [p.Customer.ProgramAccount].[CustomerTypeId], [p.Customer.ProgramAccount].[DefaultPricing],
[p.Customer.ProgramAccount].[FabricSystemFreight], [p.Customer.ProgramAccount].[IsActive], [p.Customer.ProgramAccount].[IsSpecialFreight], [p.Customer.ProgramAccount].[LocalityRepId], [p.Customer.ProgramAccount].[MinFreightCharge],
[p.Customer.ProgramAccount].[CompanyName], [p.Customer.ProgramAccount].[Notes], [p.Customer.ProgramAccount].[PrimaryBusinessId], [p.Customer.ProgramAccount].[ProgramAccountId], [p.Customer.ProgramAccount].[Prospect],
[p.Customer.ProgramAccount].[ShippingDollarsThreshold], [p.Customer.ProgramAccount].[ShippingMilesThreshold], [p.Customer.ProgramAccount].[SpecialNote], [p.Customer.ProgramAccount].[SpecialPlantInstruction],
[p.Customer.ProgramAccount].[UpdatedById], [p.Customer.ProgramAccount].[UpdatedOn], [p.Customer.ProgramAccount].[Xq1ProgGroupName], [p.Customer.ProgramAccount].[Xq1SrNo], [p.Customer.ProgramAccount].[XqCustomerNumber],
[p.Customer.CustomerType].[Id], [p.Customer.CustomerType].[Code], [p.Customer.CustomerType].[Name]
FROM [Projects] AS [p]
LEFT JOIN [DataMigrationLogs] AS [p.DataMigrationLog] ON [p].[DataMigrationLogId] = [p.DataMigrationLog].[Id]
INNER JOIN [Security].[XQUsers] AS [p.UpdatedBy] ON [p].[UpdatedById] = [p.UpdatedBy].[Id]
INNER JOIN [Security].[XQUsers] AS [p.CreatedBy] ON [p].[CreatedById] = [p.CreatedBy].[Id]
INNER JOIN [ProjectStatuses] AS [p.Status] ON [p].[StatusId] = [p.Status].[Id]
LEFT JOIN [Customers] AS [p.Customer] ON [p].[CustomerId] = [p.Customer].[Id]
LEFT JOIN (
SELECT [p.Customer.LocalityRep].*
FROM [Security].[XQUsers] AS [p.Customer.LocalityRep]
WHERE [p.Customer.LocalityRep].[Discriminator] = N'SALES_SP'
) AS [t] ON [p.Customer].[LocalityRepId] = [t].[Id]
LEFT JOIN [Customers] AS [p.Customer.ProgramAccount] ON [p.Customer].[ProgramAccountId] = [p.Customer.ProgramAccount].[Id]
LEFT JOIN (
SELECT [p.Customer.ProgramAccount.LocalityRep].*
FROM [Security].[XQUsers] AS [p.Customer.ProgramAccount.LocalityRep]
WHERE [p.Customer.ProgramAccount.LocalityRep].[Discriminator] = N'SALES_SP'
) AS [t0] ON [p.Customer.ProgramAccount].[LocalityRepId] = [t0].[Id]
LEFT JOIN [CustomerTypes] AS [p.Customer.CustomerType] ON [p.Customer].[CustomerTypeId] = [p.Customer.CustomerType].[Id]
WHERE ([p.UpdatedBy].[Discriminator] IN (N'INT_SYSADMMIN', N'XqInternalUser', N'EXT', N'SALES_SP', N'XqUser') AND [p.CreatedBy].[Discriminator] IN (N'INT_SYSADMMIN', N'XqInternalUser', N'EXT', N'SALES_SP', N'XqUser')) AND (EXISTS (
SELECT 1
FROM [Security].[BusinessUserRegions] AS [r]
WHERE [r].[RegionId] IN (CAST(6 AS bigint), CAST(8 AS bigint), CAST(9 AS bigint)) AND ([t].[Id] = [r].[BusinessUserId])) OR ([p.Customer].[ProgramAccountId] IS NOT NULL AND EXISTS (
SELECT 1
FROM [Security].[BusinessUserRegions] AS [r0]
WHERE [r0].[RegionId] IN (CAST(6 AS bigint), CAST(8 AS bigint), CAST(9 AS bigint)) AND ([t0].[Id] = [r0].[BusinessUserId]))))
ORDER BY [p].[CreatedOn]
It takes more than 30 seconds to execute and Azure SQL Server has query timeout of maximum 30 second, so it results in exception
System.Data.SqlClient.SqlException (0x80131904): Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
From my layman's understanding about SQL I think SQL us fetching the entire dataset for sorting and then trying to do pagination and hence it's taking more than 30 seconds. Please correct me if I am wrong.
I need to sort the result in query in self otherwise it will not provide the desired result set.
Is their any way to deal with this situation.
Any help is highly appreciated.
SELECT two.WorkOrderID,
te.EquipmentID,
tm.MaterialID,
CodMaterial,
MaterialName,
EquipmentName,
GoalPriceEnergyPerUnit,
pt.ParameterTypeID
FROM T_WorkOrders two
LEFT OUTER JOIN T_Equipment te
ON te.EquipmentID = two.EquipmentID
LEFT OUTER JOIN T_Materials tm
ON tm.MaterialID = two.MaterialID
LEFT JOIN T_Parameter_Type pt
ON ea.ParameterTypeID = pt.ParameterTypeID
WHERE WorkOrderNumber = 2
AND tm.MaterialID = 417
AND te.EquipmentID = 1076
AND ea.ParameterTypeID = 4918
Just completing,what was said in comments ..
you have defined table aliases in your query like below
T_Equipment te
T_Materials tm
T_Parameter_Type pt
and finally you are referring a column with an alias that is not in any one of the above..
ea.ParameterTypeID
so refer that column using the alias ,for which the table belongs
You are not connecting your tables because you are using the wrong alias. Try looking at your tables
T_Equipment
MaterialID
T_WorkOrders
to see which one has a column that you can 'connect' to with ParameterTypeID.
I can't help you more than that as you haven't given enough information about your tables and what you really want to do. But here's a try anyway
SELECT two.WorkOrderID,
te.EquipmentID,
tm.MaterialID,
CodMaterial,
MaterialName,
EquipmentName,
GoalPriceEnergyPerUnit,
pt.ParameterTypeID
FROM T_WorkOrders two
LEFT JOIN T_Equipment te
ON te.EquipmentID = two.EquipmentID
LEFT JOIN T_Materials tm
ON tm.MaterialID = two.MaterialID
LEFT JOIN T_Parameter_Type pt
( -- this is not correct syntax
ON te.ParameterTypeID = pt.ParameterTypeID
ON tm.ParameterTypeID = pt.ParameterTypeID
ON two.ParameterTypeID = pt.ParameterTypeID
)
WHERE WorkOrderNumber = 2
AND tm.MaterialID = 417
AND te.EquipmentID = 1076
AND ea.ParameterTypeID = 4918
Code above is example of possible uses of LEFT JOIN and ON where I'm connecting different tables. Now as you can see in the comment, this will not work because the code has two ON-s extra. All I'm trying to do here is show you possible combinations that you have maybe missed.
Also, the code you have given in comments to #TheGameiswar is not correct.
SELECT two.WorkOrderID,
te.EquipmentID,
tm.MaterialID,
CodMaterial,
MaterialName,
EquipmentName,
GoalPriceEnergyPerUnit,
pt.ParameterTypeID
FROM T_WorkOrders two
LEFT JOIN T_Equipment te
ON te.EquipmentID = two.EquipmentID
LEFT JOIN T_Materials tm
ON tm.MaterialID = two.MaterialID
LEFT JOIN T_Parameter_Type pt
--ON is missing, maybe something like
-- pt.SomeID = two.SomeID
LEFT JOIN T_Equipment_Address ea
ON ea.ParameterTypeID=pt.ParameterTypeID
WHERE WorkOrderNumber=2
AND tm.MaterialID = 417
AND te.EquipmentID = 1076
AND pt.ParameterTypeID = 4918
It's missing ON statement as shown in the code comment. Try to fix that. Also using Microsoft SQL Server Management Studio helps a lot in writing sql code because it will warn you if you are missing something. Also, I sometimes use visual reminders for JOIN just to be sure what am I doing, often this one from codinghorror.
select sum(l.coins) - sum(t.coins) as total
from luxx_getaway_2016_coins l
join thrive_rewards_redeemed t
on l.consid = t.guideid
where l.consid = 24969 and t.harvestyear = 1516
Hello all. I am attempting to grab an updated total using the query above. The problem I'm having is that the total of these sums totals out to well above what it should be. I'm unsure of what I'm doing wrong. We're using Azure SQL Database and I've used RazorSQL and SSMS 2012 to run this query with identical results. Any help is appreciated. Please feel free to ask for clarification.
A simple solution to your duplication problem:
Select
(select sum(l.coins)
from luxx_getaway_2016_coins l
where l.consid = 24969)
-
(select sum(t.coins)
from thrive_rewards_redeemed t
where t.guideid = 24969
and t.harvestyear = 1516)
For the more general case:
; With A as (select consid, sum(l.coins) as TotA
from luxx_getaway_2016_coins l
group by consid)
, B as (select guideid, sum(t.coins) as TotB
from thrive_rewards_redeemed t
where t.harvestyear = 1516
group by guideID)
Select a.consid, TotA - TotB as Total
from A
inner join B
on a.Consid = b.GuideID
I am trying to translate a QueryExpression that is in some existing code into a T-SQL select statement.
I've run across the following statement and I'm having trouble understanding what they mean by a Natural Join:
linkEntity1.JoinOperator = JoinOperator.Natural;
Would this be equivalent to an Inner Join in T-SQL? Googling has not been much help.
Here's the rest of the QueryExpression Code:
QueryExpression query = new QueryExpression();
query.EntityName = "showinfo";
ColumnSet columns = new ColumnSet();
columns.Attributes = new String[] { "company" };
query.ColumnSet = columns;
query.Criteria = new FilterExpression();
query.Criteria.FilterOperator = LogicalOperator.And;
ConditionExpression condition1 = new ConditionExpression();
condition1.AttributeName = "company";
condition1.Operator = ConditionOperator.NotNull;
query.Criteria.Conditions = new ConditionExpression[] { condition1 };
LinkEntity linkEntity1 = new LinkEntity();
linkEntity1.JoinOperator = JoinOperator.Natural;
linkEntity1.LinkFromEntityName = "show";
linkEntity1.LinkFromAttributeName = "showid";
linkEntity1.LinkToEntityName = "showintegration";
linkEntity1.LinkToAttributeName = "showcode";
linkEntity1.LinkCriteria = new FilterExpression();
linkEntity1.LinkCriteria.FilterOperator = LogicalOperator.And;
ConditionExpression condition2 = new ConditionExpression();
condition2.AttributeName = "showend";
condition2.Operator = ConditionOperator.Null;
linkEntity1.LinkCriteria.Conditions = new ConditionExpression[] { condition2 };
query.LinkEntities = new LinkEntity[] { linkEntity1 };
There is no equivalent in SQL Server of a natural join where table intersect is based on column names by the RDBMS.
I'm glad of that because it is at best ambiguous and at worst dangerous. JOINs should be explicit. Examples why:
having a InsertedBy column in both tables (quite common): should we have to prefix with the table name to remove ambiguity?
future DDL that add columns that change JOIN semantics
See
Natural join in SQL Server
SQL Server - lack of NATURAL JOIN / x JOIN y USING(field)
Edit:
It looks like natural join means "don't repeat the column in the output" (like USING in MySQL would do) according to the JoinOperator Enumeration.
If I understand this (debatable!) it's misleading. Especially when I read the "LeftOuter" narrative..
A natural join compares all columns in the two tables that have the same column names. It's equivalent to an inner join with the matching columns explicitly listed.
Yes - the natural join is inner join - so you can write:
select * from tab1, tab2 where tab1.col1 = tab2.col1
as
select * from tab1 inner join tab2 on tab1.col1 = tab2.col1
Is there a better way to write the following simple SQL Server 2005 update statement? This just seems a bit messy inefficient.
UPDATE QuotationItem
SET Recurring_Cost =
(SELECT TOP (1) Recurring_Cost
FROM Products
WHERE (Remote_ID = QuotationItem.Product_ID))
WHERE (Quotation_ID = 115)
Thanks,
Nick
How About using a join
UPDATE QuotationItem
SET Recurring_Cost = p.recurring_cost
FROM QuotationItem q join Products p on q.Product_ID = p.Remote_ID
WHERE q.Quotation_ID = 115
Is your TOP 1 really needed? If it is, since you don't specify an ordering, you've got pretty random results from your query anyway! If it is not really necessary, this will do:
UPDATE q
SET Recurring_Cost = p.RecurringCost
FROM QuotationItem q
INNER JOIN
Products p
ON p.Remote_ID = q.Product_ID
WHERE q.Quotation_ID = 115